Album generating apparatus, album generating method and computer readable medium

ABSTRACT

There is provided an album generating apparatus for generating an album in which a desirable combination of images is disposed in one and same page. The album generating apparatus has an image storing section for storing a plurality of images, an object sampling section for sampling objects contained in each image stored in the image storing section, a relationship judging section for judging that a relationship of the first object with the second object is high when a number of images containing the first and second objects sampled by the object sampling section is larger than a reference number set in advance in the images stored in the image storing section, a main image selecting section for selecting a main image that is to be a primary object in the album out of the plurality of images stored in the image storing section, a subsidiary image selecting section for selecting the image containing the second object that has been judged by the relationship judging section to be highly related with the first object contained in the main image selected as a subsidiary and a layout determining section for laying out the main image and the subsidiary image in the album based on contents of the main image and the subsidiary image.

TECHNICAL FIELD

The present invention relates to an album generating apparatus, an albumgenerating method and a computer readable medium. More specifically, theinvention relates to an album generating apparatus and an albumgenerating method for generating an album and to a computer readablemedium storing thereon a program for the album generating apparatus. Thepresent application claims priority from Japanese Patent ApplicationsNos. JP 2005-074091 filed in JP on Mar. 15, 2005 JP 2006-13144 filed inJP on Jan. 20, 2006, the contents of which are incorporated herein byreference.

BACKGROUND ART

There has been known an album generating system for generating an albumlaid out so as to dispose a map of traveling spots at a center of ascreen and so that a user can see a relationship between the map andimages disposed around the map by connecting the images and imagecapturing positions on the map by lines based on tag information such asinformation on position, image capturing time and others tagged to dataof images captured at the traveling spots as disclosed in JapanesePatent Laid-Open No. 1998-126731 for example.

However, there has been a case that an image containing a person and animage containing an other person such as a tourist who is not related tothat person at all are stored in one and same page of an album when thealbum is generated by the technology of JP Laid-Open No. 1998-126721.Such album cannot be what reflects the intention of the cameraman and iswhat spoils interest of the cameraman. An album recording a travel forexample is desirable to be generated so that images containing friendswho traveled together are laid out in one and same page.

DISCLOSURE OF THE INVENTION

However, there has been a problem that it takes time for a generator ofthe album to generate such album because the generator is required tolay out the images to each page by judging human relationship and othersof the persons, i.e., the subjects, while seeing the contents of theimages.

Accordingly, it is an object of the invention to provide an albumgenerating apparatus, an album generating method and a program that arecapable of solving the above-mentioned problems. This object may beachieved through the combination of features described in independentclaims of the invention. Dependent claims thereof specify preferableembodiments of the invention.

According to a first aspect of the invention, an album generatingapparatus has an image storing section for storing a plurality ofimages, an object sampling section for sampling a plurality of objectscontained in each of the plurality of images stored in the image storingsection, a relationship judging section for judging that a relationshipof the first object with the second object is high when a number ofimages containing the first and second objects sampled by the objectsampling section is larger than a reference number set in advance in theplurality of objects stored in the image storing section, a main imageselecting section for selecting a main image that is to be a primaryobject in the album out of the plurality of images stored in the imagestoring section, a subsidiary image selecting section for selecting theimage containing the second object that has been judged by therelationship judging section to be highly related with the first objectcontained in the main image selected by the main image selecting sectionas a subsidiary image to be subsidiary to the main image in the albumselected by the main image selecting section and a layout determiningsection for laying out the main image selected by the main imageselecting section and the subsidiary image selected by the subsidiaryimage selecting section in the album based on contents of the main imageselected by the main image selecting section and the subsidiary imageselected by the subsidiary image selecting section.

The album generating apparatus may further include a distancecalculating section for calculating distance between the first andsecond objects in the image containing the first and second objects andthe relationship judging section may judge that the relationship of thefirst object with the second object is high when the distance calculatedby the distance calculating section is short.

The album generating apparatus may further include a distancecalculating section for calculating distance between the first andsecond objects in actual space and the relationship judging section mayjudge that the relationship of the first object with the second objectis high when the distance calculated by the distance calculating sectionis short.

The album generating apparatus may further include a person recognizingsection for recognizing whether or not an object sampled by the objectsampling section is a person and the relationship judging section mayjudge a relationship of objects recognized as persons by the personrecognizing section to be higher than a relationship of objectsrecognized as not persons by the person recognizing section.

According to a second aspect of the invention, an album generatingmethod has an image storing step of storing a plurality of images, anobject sampling step of sampling a plurality of objects contained ineach of the plurality of images stored in the image storing step, arelationship judging step of judging that relationship of the firstobject with the second object is high when a number of images containingthe first and second objects sampled in the object sampling step islarger than a reference number set in advance in the plurality ofobjects stored in the image storing step, a main image selecting step ofselecting a main image that is to be a primary object in the album outof the plurality of images stored in the image storing step, asubsidiary image selecting step of selecting the image containing thesecond object that has been judged by the relationship judging sectionto be highly related with the first object contained in the main imageselected in the main image selecting step as a subsidiary image to besubsidiary to the main image in the album selected in the main imageselecting step and a layout determining step of laying out the mainimage selected in the main image selecting step and the subsidiary imageselected in the subsidiary image selecting step in the album based oncontents of the main image selected in the main image selecting step andthe subsidiary image selected in the subsidiary image selecting step.

According to a third aspect of the invention, a program for an albumgenerating apparatus for generating an album operates the albumgenerating apparatus so as to function as an image storing section forstoring a plurality of images, an object sampling section for sampling aplurality of objects contained in each of the plurality of images storedin the image storing section, a relationship judging section for judgingthat the relationship of the first object with the second object is highwhen a number of images containing the first and second objects sampledby the object sampling section is larger than a reference number set inadvance in the plurality of objects stored in the image storing section,a main image selecting section for selecting a main image that is to bea primary object in the album out of the plurality of images stored inthe image storing section, a subsidiary image selecting section forselecting the image containing the second object that has been judged bythe relationship judging section to be highly related with the firstobject contained in the main image selected by the main image selectingsection as a subsidiary image to be subsidiary to the main image in thealbum selected by the main image selecting section and a layoutdetermining section for laying out the main image selected by the mainimage selecting section and the subsidiary image selected by thesubsidiary image selecting section in the album based on contents of themain image selected by the main image selecting section and thesubsidiary image selected by the subsidiary image selecting section.

According to a fourth aspect of the invention, an album generatingapparatus has an image storing section for storing a plurality ofimages, a main image selecting section for selecting a main image thatis to be a primary image in the album out of the plurality of imagesstored in the image storing section, a relationship judging section forjudging a relationship of the main image selected by the main imageselecting section with the plurality of images stored in the imagestoring section based on contents of the plurality of images stored inthe image storing section, a subsidiary image selecting section forselecting an image, out of the plurality of images stored in the imagestoring section, judged by the relationship judging section to be highlyrelated with the main image selected by the main image selecting sectionas a subsidiary image that is to be an image subsidiary to the mainimage selected by the main image selecting section in the album and alayout determining section for laying out the main image selected by themain image selecting section and the subsidiary image selected by thesubsidiary image selecting section in one and same page in the albumbased on the contents of the main image selected by the main imageselecting section and the subsidiary image selected by the subsidiaryimage selecting section.

The album generating apparatus may further include a template storingsection for storing a judging condition for judging the degree ofrelationship between the main image and the subsidiary image and therelationship judging section may judge the degree of relationshipbetween the main image and the plurality of images stored in the imagestoring section based on the judging conditions stored in the templatestoring section.

The template storing section may store a judging condition of highlyjudging the relationship of the main image with an image whose imagecapturing time is closer to that of the main image and the relationshipjudging section may highly judge the relationship of the image whoseimage capturing time is closer to that of the main image.

The album generating apparatus may further include a compositionalsimilarity calculating section for calculating the degree of similarityof composition of the images based on composition of subjects containedin the plurality of images stored in the image storing section, and thetemplate storing section may store a judging condition of highly judgingthe relationship of the main image with the image whose composition issimilar to that of the main image and the relationship judging sectionmay highly judge the relationship of the main image with the image whosecomposition is calculated to be highly similar to that of the main imageby the compositional similarity calculating section.

The album generating apparatus may further include a color similaritycalculating section for judging the degree of similarity of colors ofthe plurality of images stored in the image storing section based ondistributions of colors contained therein, and the template storingsection may store a judging condition of highly judging the relationshipof the main image with an image containing color therein is similar tothat of the main image and the relationship judging section may highlyjudge the relationship of the main image with the image whose color iscalculated to be highly similar to that of the main image by the colorsimilarity calculating section.

The album generating apparatus may further include a similarity-of imagecapturing environment calculating section for calculating similarity ofimage capturing environments in which the plurality of images stored inthe image storing section were captured, and the template storingsection may store a judging condition of highly judging the relationshipof the main image with the image whose image capturing environment issimilar to that of the main image and the relationship judging sectionmay highly judge the relationship of the main image with the image whoseimage capturing environment is calculated to be highly similar to thatof the main image by the similarity-of image capturing environmentcalculating section.

The album generating apparatus may further include an image capturingplace specifying section for specifying whether the places where theplurality of images stored in the image storing section were capturedare indoor or outdoor based on contents of the plurality of imagesstored in the image storing section, and the template storing sectionmay store a judging condition of highly judging the relationship of themain image with an image whose image capturing place is similar to thatof the main image and the relationship judging section may highlycalculate the relationship of the image, with the main image, whoseimage capturing place is specified to be the same with that of the mainimage by the image capturing place specifying section more than therelationship of an image, with the main image, whose image capturingplace is specified to be different from that of the main image by theimage capturing place specifying section.

The album generating apparatus may further include a time similaritycalculating section for judging closeness of image capturing times ofthe plurality of images stored in the image storing section, and thetemplate storing section may store a judging condition of highly judgingthe relationship of the main image with the image whose image capturingtime is close to that of the main image and the relationship judgingsection may highly judge the relationship of the main image with theimage whose image capturing time is calculated to be closer to that ofthe main image by the time similarity calculating section.

According to a fifth aspect of the invention, an album generating methodhas an image storing step of storing a plurality of images, a main imageselecting step of selecting a main image that is to be a primary imagein the album out of the plurality of images stored in the image storingstep, a relationship judging step of judging a relationship of the mainimage selected in the main image selecting step with the plurality ofimages stored in the image storing step based on contents of theplurality of images stored in the image storing step, a subsidiary imageselecting step of selecting an image, out of the plurality of imagesstored in the image storing step, judged by the relationship judgingstep to be highly related with the main image selected in the main imageselecting step as a subsidiary image that is to be an image subsidiaryto the main image selected in the main image selecting step in the albumand a layout determining step of laying out the main image selected inthe main image selecting step and the subsidiary image selected in thesubsidiary image selecting step in one and same page in the album basedon the contents of the main image selected in the main image selectingstep and the subsidiary image selected in the subsidiary image selectingstep.

According to a sixth aspect of the invention, a program for an albumgenerating apparatus for generating an album operates the albumgenerating apparatus so as to function as an image storing section forstoring a plurality of images, a main image selecting section forselecting a main image that is to be a primary image in the album out ofthe plurality of images stored in the image storing section, arelationship judging section for judging a relationship of the mainimage selected by the main image selecting section with the plurality ofimages stored in the image storing section based on contents of theplurality of images stored in the image storing section, a subsidiaryimage selecting section for selecting an image, out of the plurality ofimages stored in the image storing section, judged by the relationshipjudging section to be highly related with the main image selected by themain image selecting section as a subsidiary image that is to be animage subsidiary to the main image selected by the main image selectingsection in the album and a layout determining section for laying out themain image selected by the main image selecting section and thesubsidiary image selected by the subsidiary image selecting section inone and same page in the album based on the contents of the main imageselected by the main image selecting section and the subsidiary imageselected by the subsidiary image selecting section.

It is noted that the summary of the invention described above does notnecessarily describe all necessary features of the invention. Theinvention may also be a sub-combination of the features described above.

The invention can provide the album in which images of a desirablecombination are disposed in a same page.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a drawing showing one exemplary environment for using an albumgenerating apparatus.

FIG. 2 is a drawing showing one exemplary layout result of a page of analbum.

FIG. 3 is a block diagram showing one exemplary block configuration ofthe album generating apparatus.

FIG. 4 is a flowchart showing one exemplary processing flow forgenerating the album.

FIG. 5 is a flowchart showing one exemplary processing flow for sortingimages.

FIG. 6 is a drawing showing one exemplary case when it is desirable toallocate images per page by a distribution of image capturing positions.

FIG. 7 is a drawing showing one exemplary case when it is desirable toallocate images per page by a distribution of image capturing times.

FIG. 8 shows one exemplary method for judging importance of imagecapturing position.

FIG. 9 is a chart showing one exemplary allocation of images per pagebased on the distribution of image capturing times.

FIG. 10 is a chart showing one exemplary allocation of images per pagebased on the distribution of image capturing positions.

FIG. 11 is a flowchart showing one exemplary processing flow forselecting a main image.

FIG. 12 is a drawing showing one exemplary method for calculating amomentum of an object.

FIG. 13 is a drawing showing one exemplary method for judging importanceof an object.

FIG. 14 is a flowchart showing one exemplary processing flow forselecting a subsidiary image.

FIG. 15 is a flowchart showing an other exemplary processing flow forselecting a subsidiary image.

FIG. 16 is a flowchart showing a still other exemplary processing flowfor selecting a subsidiary image.

FIG. 17 is a flowchart showing one exemplary processing flow of aspecific subject information getting section.

FIG. 18 shows one exemplary contents of specific subject information.

FIG. 19 shows exemplary templates.

FIG. 20 shows exemplary layout results.

FIG. 21 is a table showing one exemplary composite information ofimages.

FIG. 22 is a table showing one exemplary method of a characteristiccolor sampling section in sampling a characteristic color.

FIG. 23 is a diagram showing one exemplary directory configuration of analbum information file recorded in a recording medium.

FIG. 24 is a block diagram showing one exemplary block configuration ofan other embodiment of the album generating apparatus.

FIG. 25 is a drawing showing one exemplary posterization image.

FIG. 26 shows one exemplary disposition of the posterization image.

FIG. 27 is a graph showing one exemplary distribution of luminance ofimages.

FIG. 28 is a table showing one exemplary characteristic value of theposterization image.

FIG. 29 shows one exemplary judging condition stored in a templatestoring section.

FIG. 30 is a block diagram showing one exemplary hardware configurationof a computer of the album generating apparatus.

BEST MODE FOR CARRYING OUT THE INVENTION

The invention will now be described based on preferred embodiments,which do not intend to limit the scope of the invention, but exemplifythe invention. All of the features and the combinations thereofdescribed in the embodiments are not necessarily essential to theinvention.

FIG. 1 shows one exemplary environment for using an album generatingapparatus 100 according to one embodiment of the invention. When a user180 presses a release button to capture an image, an image capturingdevice 110 records image capturing time and image capturing position onthe captured image by tagging as tag information. The album generatingapparatus 100 receives the image from the image capturing device 110 viaa recording medium such as a semiconductor memory and a communicationline 150 such as Internet and radio LAN. The album generating apparatus100 selects and lays out images out of the received images to output asan album through a printer or on a display.

The album generating apparatus 100 allocates the images to each page ofthe album so that the images captured at times close to each other orcaptured at positions close to each other are stored in each page of thealbum. When a distribution of image capturing positions where the imagewere captured is geographically deviated at this time, the albumgenerating apparatus 100 allocates the images so that the images whoseimage capturing positions are close to each other are stored in eachpage. When the distribution is not geographically deviated, the albumgenerating apparatus 100 allocates the images so that the images whoseimage capturing times are close to each other are stored in each page.

Then, the album generating apparatus 100 selects an image capturing asubject that frequently appears in the plurality of images allocated toeach page as a main image representing each page. Or, the albumgenerating apparatus 100 selects an image in which a subject is moremotional in the plurality of images allocated to each page as a mainimage. Then, the album generating apparatus 100 selects an imagecapturing a subject related to the subject in the main image, such as afriend of the person in the main image of each page, as a subsidiaryimage to be disposed around the main image. Or, the album generatingapparatus 100 selects an image capturing a subject other than the personcaptured in the main image of each page as a subsidiary image. Then, thealbum generating apparatus 100 outputs the album by laying out such thatthe main image is enlarged and disposed at the center rather than thesubsidiary image in each page of the album.

It is noted that the album generating apparatus 100 may be an apparatusfor generating and printing the album containing a plurality of imagesor may be an apparatus for displaying the plurality of imagessimultaneously on a screen. For example, the album generating apparatus100 may be a personal computer, an electronic album, HDTV or a printer.

It is noted that the album generating apparatus 100 may receive theimages captured by the image capturing device 110 by means of media suchas a semiconductor memory in which such images are recorded. The albumgenerating apparatus 100 may also record the images allocated to eachpage of the album in a recording medium 160 such as a CD-ROM per page.At this time, the album generating apparatus 100 may record the imagestogether with layout information of each image in each page to arecording medium 160. The data recorded in the recording medium 160 bythe album generating apparatus 100 is used in ordering or printing thealbum or in appreciating through the electronic album. It is noted thatthe album generating apparatus 100 may be a kiosk terminal installed inphoto-shops, train stations and the like.

The album generating apparatus 100 described above can generate thealbum from which the appreciator would feel no sense of incompatibilitybecause it allocates the images whose image capturing positions or imagecapturing times are close to each other to the same page. The albumgenerating apparatus 100 also lays out an image that contains a personwho appears in many more images or a person who is in larger action as asubject for example as a main image, so that it can generate the albumin which a characteristic image is more highlighted. Still more, theimages may be combined adequately in each page because the imagescontaining the persons deeply related with the person in the main imageare automatically selected in each page. Therefore, the user 180 canhave the album without taking time.

FIG. 2 shows one exemplary layout result 300 in a page of the album. Themain image referred to in this embodiment means an image that isintended to impress the appreciator most in the page. For example, themain image may be a most highlighted image among a plurality of imagesdisposed in a page. Specifically, the main image may be an image whosesize is bigger than the other images, an image disposed to the frontside as compared to the other images or an image disposed more to thecenter. Beside that, the main image may be an image highlighted by aframe around thereof or an image in which a visual effect has beenapplied to a subject. Still more, the subsidiary image of the embodimentmay be an image smaller than the main image or an image disposed off thecenter. The example shown in FIG. 2 shows a case when an image 301 is amain image and images 302, 303 and 304 are subsidiary images. It isnoted that the album generating apparatus 100 may lay out the images bytrimming a specific subject out of the image. An image 305 is oneexemplary image laid out after trimming a specific subject.

Preferably, each page has each own theme. The theme may be contentsappealing to the appreciator such as scenes of a specific place likebeauty of a park, moves of emotion of a specific person in a certainplace or states of a specific object such as loveliness of a dog. In anycase, the theme of the page is determined by the main image. When themain image is once determined, the subsidiary images having the sametheme may be selected.

It is noted that the album generating apparatus 100 may have a pluralityof templates defining image frames into which images are fitted within apage and may generate the album by fitting the images into such imageframes defined by the templates. In this case, an image frame for a mainimage into which the main image is fitted may be determined in advancein such template. Still more, the album generating apparatus 100 mayjudge importance of the image frame into which the main image is to befitted based on position, size and depth of the image frame and may fitthe main image into the image frame whose weight is high. Specifically,the album generating apparatus 100 may set the position, size and depthof the image frame as variables, may weight each variable correspondingto a degree of impression given to the appreciator and may judge theimportance of the image frame based on the variable. It is noted thatthe album generating apparatus 100 may have templates defining imageframes having a shape of image, inclination of image and the like asparameters, e.g., an image frame into which the image 303 is fitted inthe figure.

It is noted that the page referred to in this embodiment may be a spreadpage of the album or may be one page of the spread of the album. Stillmore, when the album generating apparatus 100 is an apparatus thatoutputs images as an album on a displaying device such as a displayunit, the page may be a display area in which a plurality of images isdisplayed on the displaying device in the same time.

FIG. 3 is a block diagram showing one exemplary block configuration ofthe album generating apparatus 100. The album generating apparatus 100has an image storing section 202, an object sampling section 206, anumber-of-object-image calculating section 208, a layout determiningsection 210, a main image selecting section 212, a subsidiary imageselecting section 222, a trimming section 224, a person recognizingsection 230, a momentum calculating section 232, an area-to-contourratio calculating section 234, the image sorting section 240, asimilarity calculating section 242, an distribution-of-image capturingtime calculating section 252, an distribution-of-image capturingposition calculating section 254, a importance judging section 260, arelationship judging section 272, a distance calculating section 274, aspecific subject information getting section 290, an image-of-mainperson selecting section 296, a characteristic color sampling section226, a template coloring determining section 228, a template storingsection 214, a composite information storing section 216, a light sourcespecifying section 256, a compositional similarity calculating section258 and an album information recording section 270. The image sortingsection 240 has an image capturing time sorting section 262, an imagecapturing position sorting section 264 and a main subject sortingsection 266. The layout determining section 210 has a dispositiondetermining section 282 and a size determining section 284. The specificsubject information getting section 290 has a main person inputtingsection 292 and a main person specifying section 294.

The image storing section 202 stores a plurality of images. The imagestoring section 202 also stores the images by correlating with theirimage capturing time and image capturing position. Specifically, theimage storing section 202 records the image capturing time and imagecapturing position as tag information of the image at the time when theimage capturing device 110 captured the image. Then, the albumgenerating apparatus 100 stores the image in which the image capturingtime and image capturing position are recorded by receiving from theimage capturing device 110. It is noted that a final image is oneexample of the image.

The distribution-of-image capturing position calculating section 254calculates a distribution of image capturing positions of the pluralityof images stored in the image storing section 202 based on the imagecapturing positions of the plurality of images stored in the imagestoring section 202. The distribution-of-image capturing timecalculating section 252 also calculates a distribution of imagecapturing times of the plurality of images stored in the image storingsection 202 based on the image capturing times of the plurality ofimages stored in the image storing section 202.

The image sorting section 240 sorts the plurality of images stored inthe image storing section 202 per event based on environment in whichthe image was captured. The event may be image capturing time or imagecapturing position for example. Specifically, the image sorting section240 sorts the plurality of images stored in the image storing section202 per page in the album in accordance to the image capturing times ofthe images. Still more, the image sorting section 240 sorts theplurality of images stored in the image storing section 202 per page inthe album in accordance to the image capturing positions of the images.

Specifically, the image sorting section 240 sorts the plurality ofimages stored in the image storing section 202 per page in the albumbased on the distribution of image capturing positions calculated by thedistribution-of-image capturing position calculating section 254. Theimage sorting section 240 also sorts the plurality of images stored inthe image storing section 202 per page in the album based on thedistribution of image capturing times calculated by thedistribution-of-image capturing time calculating section 252.

At this time, the image sorting section 240 sorts the plurality ofimages stored in the image storing section 202 so that a number ofimages sorted into each page of the album becomes almost equal based onthe distribution of image capturing times calculated by thedistribution-of-image capturing time calculating section 252. The imagesorting section 240 also sorts the plurality of images stored in theimage storing section 202 so that a number of images sorted into eachpage of the album becomes almost equal based on the distribution ofimage capturing positions calculated by the distribution-of-imagecapturing position calculating section 254. Then, the layout determiningsection 210 lays out the images sorted by the image sorting section 240per page to each page of the album.

The main image selecting section 212 selects a main image that is to bethe primary image in the album out of the plurality of images stored inthe image storing section 202. Then, based on the contents of the mainimage selected by the main image selecting section 212, the subsidiaryimage selecting section 222 selects subsidiary images which becomeimages subsidiary to the main image selected by the main image selectingsection 212 in the album out of the plurality of images stored in theimage storing section 202. Specifically, the main image selectingsection 212 selects each main image per event sorted by the imagesorting section 240. Then, the subsidiary image selecting section 222selects each subsidiary image per event sorted by the image sortingsection 240.

More specifically, the main image selecting section 212 selects the mainimage that is to be the primary image in a predetermined page in thealbum out of the images sorted by the image sorting section 240 to thepredetermined page. Then, based on the contents of the main imageselected by the main image selecting section 212, the subsidiary imageselecting section 222 selects the subsidiary images to be subsidiary tothe main image selected by the main image selecting section 212 in thepredetermined page of the album out of the images sorted by the imagesorting section 240 to the predetermined page. Specifically, based onthe contents of the main image selected by the main image selectingsection 212 and of the subsidiary images selected by the subsidiaryimage selecting section 222, the layout determining section 210 lays outthe main image selected by the main image selecting section 212 and thesubsidiary images selected by the subsidiary image selecting section 222to the predetermined page in the album.

More specifically, based on the contents of the main image selected bythe main image selecting section 212 and of the subsidiary imagesselected by the subsidiary image selecting section 222, the layoutdetermining section 210 determines the layout of the main image selectedby the main image selecting section 212 and the subsidiary imagesselected by the subsidiary image selecting section 222 in the album.Specifically, the disposition determining section 282 determinesdisposition of the main image selected by the main image selectingsection 212 and the subsidiary images selected by the subsidiary imageselecting section 222 in the album. The size determining section 284determines the size of the main image selected by the main imageselecting section 212 and of the subsidiary images selected by thesubsidiary image selecting section 222 in the album. More specifically,the size determining section 284 determines the size of the main imageselected by the main image selecting section 212 to be larger than thatof the subsidiary images selected by the subsidiary image selectingsection 222.

The similarity calculating section 242 calculates similarity of the mainimage selected by the main image selecting section 212 with each of theother images sorted into the same event with the main image selected bythe main image selecting section 212. Then, the subsidiary imageselecting section 222 selects the images whose similarity with the mainimage selected by the main image selecting section 212 is small as thesubsidiary images per event sorted by the image sorting section 240. Thesimilarity may be those determined by such indices as a focal distance,image capturing time, image capturing position or a hue of images whencaptured.

Therefore, it becomes possible to prevent the images stored in the samepage from becoming only images captured in the same image capturingplace or only images capturing a distant place for example.

Beside that, the subsidiary image selecting section 222 selects imagescontaining subjects different from the subject of the main imageselected by the main image selecting section 212 as the subsidiaryimages. For example, when a subject of the main image is sea, thesubsidiary image selecting section 222 selects images containing personsas subsidiary images. Therefore, it is possible to prevent the contentsof the main image and the subsidiary images from overlapping from eachother. Still more, the subsidiary image selecting section 222 selectsimages containing persons other than the person who is the subject ofthe main image selected by the main image selecting section 212 assubsidiary images. Therefore, it is possible to prevent the images inthe same page from becoming those containing only the same person. Stillmore, when there are more images belonging to the event sorted by theimage sorting section 240, the subsidiary image selecting section 222selects more subsidiary images. Therefore, the more the cameramancaptures images with much interest, the more subsidiary images thesubsidiary image selecting section 222 lays out, so that it becomespossible to provide a desirable album for the cameraman.

The importance judging section 260 judges the importance of the imagecapturing time and image capturing position in the plurality of imagesstored in the image storing section 202. Specifically, the importancejudging section 260 judges that the image capturing position is moreimportant than the image capturing time when deviation of thedistribution of image capturing positions calculated by thedistribution-of-image capturing position calculating section 254 islarger than a reference value set in advance. When a traveling recordcaptured while traveling a tourist spot is compared with a record ofchild's growth captured in a home for example, a distribution of imagecapturing positions of the traveling record is wider than that of thegrowth record and is what captured at deviated image capturingpositions. When the importance judging section 260 detects the deviationof the image capturing positions in this case, it judges that theimportance of the image capturing position is greater than that of theimage capturing time. It is noted that the album generating apparatus100 may enable the user 180 to specify whether the image capturingposition is important or the image capturing time is important.

When the importance judging section 260 judges that the importance ofthe image capturing position is greater than that of the image capturingtime, the image capturing position sorting section 264 sorts theplurality of images stored in the image storing section 202 per page inthe album based on the image capturing positions stored in the imagestoring section 202. When the importance judging section 260 judges thatthe importance of the image capturing time is greater than that of theimage capturing position, the image capturing time sorting section 262sorts the plurality of images stored in the image storing section 202per page in the album based on the image capturing times stored in theimage storing section 202.

It is noted that the image capturing position sorting section 264 sortsthe plurality of images stored in the image storing section 202 so thatthe number of images sorted into each page of the album becomes almostequal based on the distribution of image capturing positions calculatedby the distribution-of-image capturing position calculating section 254.Or, the image capturing time sorting section 262 sorts the plurality ofimages stored in the image storing section 202 so that the number ofimages sorted into each page of the album becomes almost equal based onthe distribution of image capturing times calculated by thedistribution-of-image capturing time calculating section 252.

The main image selecting section 212 selects the main image that is tobe the primary image in the predetermined page of the album out of theimages sorted into the predetermined page by the image capturing timesorting section 262 or the image capturing position sorting section 264.The subsidiary image selecting section 222 also selects the subsidiaryimages that are to be the images subsidiary to the main image selectedby the main image selecting section 212 in the predetermined page of thealbum out of the images sorted into the predetermined page by the imagecapturing time sorting section 262 or the image capturing positionsorting section 264 based on the contents of the main image selected bythe main image selecting section 212. Then, the layout determiningsection 210 lays out the images sorted by the image capturing timesorting section 262 or the image capturing position sorting section 264per page into the respective pages of the album.

The object sampling section 206 samples a plurality of objects containedin each of the plurality of images stored in the image storing section202. Specifically, the object sampling section 206 samples contours ofthe subject by conducting edge sampling or the like to the image tosample an area of the subject as an object. Then, thenumber-of-object-image calculating section 208 calculates a number ofimages, among the plurality of images stored in the image storingsection 202, containing the same or similar object with the objectsampled by the object sampling section 206 per plurality of objectssampled by the object sampling section 206.

Then, the layout determining section 210 lays out the object containedin a larger number of images calculated by the number-of-object-imagecalculating section 208 while highlighting more in the album.Specifically, the layout determining section 210 lays out the imagecontaining the object contained in the larger number of imagescalculated by the number-of-object-image calculating section 208 in alarge size.

More specifically, the main image selecting section 212 selects theimage containing the object contained in the number of images calculatedby the number-of-object-image calculating section 208 which is largerthan the reference value set in advance out of the plurality of imagesstored in the image storing section 202 as the main image that is to bethe primary image in the album. Then, the layout determining section 210lays out the main image selected by the main image selecting section 212in highlight more than the subsidiary images selected by the subsidiaryimage selecting section 222.

Further, the trimming section 224 trims an regional image of the objectcontained in the number of images which is calculated by thenumber-of-object-image calculating section 208 and which is greater thanthe reference value set in advance. Then, the layout determining section210 lays out the regional image of the object trimmed by the trimmingsection 224 in the album.

The number-of-object-image calculating section 208 calculates a numberof images containing the object of the same type with the object sampledby the object sampling section 206 out of the plurality of images storedin the image storing section 202 per the plurality of objects sampled bythe object sampling section 206. Then, the layout determining section210 lays out the object of the type in the images whose numbercalculated by the number-of-object-image calculating section 208 isgreater while highlighting more in the album. For example, thenumber-of-object-image calculating section 208 may judge that theobjects are those of same type when their image capturing positions orimage capturing times are close to each other. Because thenumber-of-object-image calculating section 208 judges that imagescaptured by the user 180 continuously at the same place are those ofsame type and lays out in highlight in this case, the album adequatelyreflects the intention of the user 180 who captured the images.

The person recognizing section 230 recognizes whether or not the objectsampled by the object sampling section 206 is a person. The personrecognizing section 230 judges whether or not the object is a person bymeans of pattern matching by matching a pattern of contours of theobject sampled by edge sampling and the like with a pattern of the humanbody set in advance. The person recognizing section 230 also identifiesa person based on contour profile of a face, shapes of characteristicparts in a face such as eyes, nose and mouth, position of each part inthe whole face or positional relationship among the respective parts.

Then, the number-of-object-image calculating section 208 calculates anumber of images containing the same person with the person sampled bythe object sampling section 206 out of the plurality of images stored inthe image storing section 202 per plurality of persons sampled by theobject sampling section 206. Then, the layout determining section 210lays out the person contained in a larger number of images as calculatedby the number-of-object-image calculating section 208 while highlightingmore in the album. For example, the main image selecting section 212selects the image containing that person as a main image and the layoutdetermining section 210 lays out the image by highlighting as a mainimage. Therefore, the person who is contained as a subject in a largernumber of images may be laid out in highlight.

It is noted that the number-of-object-image calculating section 208 setsa criterion for judging as a same or similar object in advance inaccordance to types of object. For example, when the person recognizingsection 230 judges that an object is a person, thenumber-of-object-image calculating section 208 handles the same personas the same object. Still more, as for an object having suchcharacteristics that it is a rectangular parallelepiped and its actualsize is large, the object is judged to be a building and thenumber-of-object-image calculating section 208 handles it as a similarobject even when it is a totally different building. Thus, thenumber-of-object-image calculating section 208 can count the images inaccordance to categories desirable for the appreciator by counting theimages per person with respect to the personal subject and by countingthe images per category of building with respect to buildings and thelike.

It is noted that the image storing section 202 also stores a preliminaryimage captured in succession before or after the image stored in theimage storing section 202 by correlating with each one of the pluralityof images stored in the image storing section 202. Specifically, theimage capturing device 110 captures the preliminary image before orafter the timing when the user 180 presses the release button andrecords the preliminary image by correlating with the image captured atthe timing when the user 180 presses the release button. Then, the imagestoring section 202 stores the image and the preliminary image byreceiving from the image capturing device 110. The object samplingsection 206 samples an object contained in each of the plurality ofimages stored in the image storing section 202.

The momentum calculating section 232 calculates a momentum thatindicates a scale of motion of the object sampled by the object samplingsection 206. Specifically, the momentum calculating section 232calculates the momentum of the object sampled by the object samplingsection 206 by comparing the image stored in the image storing section202 with the preliminary image. The area-to-contour ratio calculatingsection 234 calculates length of contours of the object in the imagewith respect to an area of the object in the image. Then, the momentumcalculating section 232 calculates the momentum of the object to belarge when the calculation result of the area-to-contour ratiocalculating section 234 is large.

Then, the layout determining section 210 lays out the object whosemomentum calculated by the momentum calculating section 232 is larger byhighlighting in the album. For example, the layout determining section210 lays out the image containing the object whose momentum calculatedby the momentum calculating section 232 is large in a large size in thealbum. Specifically, the layout determining section 210 lays out theobject whose momentum calculated by the momentum calculating section 232is large as a main image. Therefore, it becomes possible to generate thealbum in which a person having a large action is highlighted.

Specifically, the main image selecting section 212 selects the imagecontaining the object whose momentum calculated by the momentumcalculating section 232 is larger than a reference value set in advanceout of the plurality of images stored in the image storing section 202as a main image that is to be a primary image in the album. Then, basedon the contents of the main image selected by the main image selectingsection 212, the subsidiary image selecting section 222 selectssubsidiary images that are to be images subsidiary to the main imageselected by the main image selecting section 212 in the album out of theplurality of images stored in the image storing section 202. Then, thelayout determining section 210 lays out the main image selected by themain image selecting section 212 by highlighting more than thesubsidiary images selected by the subsidiary image selecting section222. It is noted that the trimming section 224 may trim a regional imageof the object whose momentum calculated by the momentum calculatingsection 232 is larger than the reference value set in advance.

The distance calculating section 274 calculates a distance between afirst object and a second object in an image containing the first andsecond objects sampled by the object sampling section 206. It is notedthat the distance calculating section 274 may calculate a distancebetween the first and second objects in a real space. For example, thedistance calculating section 274 may calculate the distance between theobjects in a real space by recording a focal distance per image regiontogether with the image and by calculating position in the real spaceper object based on the focal distance per image region and position ofthe object on the image.

When a number of images containing the first and second objects sampledby the object sampling section 206 is larger than a reference number setin advance in the plurality of images stored in the image storingsection 202, the relationship judging section 272 judges that the firstobject is highly related with the second object. Still more, when thedistance calculated by the distance calculating section 274 is short,the relationship judging section 272 may judge that the first object ishighly related with the second object. It is noted that the relationshipjudging section 272 may judge the relationship between the objectsrecognized as being persons by the person recognizing section 230 to begreater than the relationship between the objects judged as being notpersons by the person recognizing section 230. Therefore, when the user180 traveled in a group of friends for example and there are many imagescontaining the friends, the relationship judging section 272 can judgethe relationship of the friends to be high.

Then, the subsidiary image selecting section 222 selects the imagescontaining the second object judged by the relationship judging section272 to be highly related with the first object contained in the mainimage selected by the main image selecting section 212 out of theplurality of images stored in the image storing section 202 as thesubsidiary images that is to be the images subsidiary to the main imageselected by the main image selecting section 212 in the album.

It is noted that the image capturing device 110 may function as thealbum generating apparatus 100. For example, the image capturing device110 may generate an album out of the plurality of images captured by theimage capturing device 110. The image capturing device 110 may displaygenerated album data on a monitor screen or may output the album data toan output device other than the image capturing device 110 via thecommunication line 150 such as Internet for example.

The specific subject information getting section 290 obtains an image ofa main person in an album. Specifically, the main person inputtingsection 292 causes one to input the image containing the main person inthe album out of the plurality of images stored in the image storingsection 202 and position of the main person in the image. For example,the main person inputting section 292 causes the user 180 to input theimage containing the main person and the position of the main person inthe image. Then, the main person specifying section 294 specifies theimage of the main person from the images containing the main person andthe position inputted through the main person inputting section 292.Then, the image-of-main person selecting section 296 selects the imagecontaining the main person out of the plurality of images stored in theimage storing section 202 by comparing the image of the main personspecified by the main person specifying section 294 with subjectscontained in the plurality of images stored in the image storing section202. Then, the main image selecting section 212 selects a main image outof the images selected by the image-of-main person selecting section296. Therefore, the album generating apparatus 100 can determine theimage containing the hero in the album specified by the user 180 as themain image.

It is noted that the main person inputting section 292 may enable one toinput an image containing a first main person and position of the firstmain person as well as an image containing a second main person andposition of the second person. Then, the main person specifying section294 specifies the images of the first main person and the second mainperson from the image containing the first main person and the positionas well as the image containing the second main person and the positioninputted through the main person inputting section 292. Theimage-of-main person selecting section 296 selects the image containingthe first main person and the image containing the second main personout of the plurality of images stored in the image storing section 202by comparing the images containing the first and second main personsspecified by the main person specifying section 294 with the subjectscontained in the plurality of images stored in the image storing section202.

Then, the main image selecting section 212 selects the main image out ofthe images containing the first main person and the images containingthe second main person selected by the image-of-main person selectingsection 296. Further, based on the contents of the main image selectedby the main image selecting section 212, the subsidiary image selectingsection 222 selects the subsidiary image that is to be subsidiary to themain image selected by the main image selecting section 212 in the albumout of the plurality of images stored in the image storing section 202.

Then, the layout determining section 210 determines layout of the mainand subsidiary images so that a number of the first main personscontained in the plurality of images laid out in one and same pagebecomes almost equal with a number of the second main persons. Thelayout determining section 210 also determines layout of the main andsubsidiary images so that an area of the first main person and an areaof the second main person contained in the page of the album containingthe first and second main persons becomes almost equal. Thereby, thealbum generating apparatus 100 can prevent one of the plurality ofpersons specified as being the heroes in the album from beinghighlighted in the page in advance.

The template storing section 214 stores templates of the albumcontaining a plurality of images. Then, the characteristic colorsampling section 226 samples a characteristic color out of a pluralityof colors contained in the main image selected by the main imageselecting section 212. Based on the characteristic color sampled by thecharacteristic color sampling section 226, the template coloringdetermining section 228 determines coloring of the template of the albumcontaining the plurality of images. Thereby, the album generatingapparatus 100 can generate the album having the coloring harmonized withthe color in the main image.

The image sorting section 240 sorts the plurality of images stored inthe image storing section 202 per page in the album in accordance to animage capturing condition in capturing the images. Specifically, theimage sorting section 240 sorts the plurality of images stored in theimage storing section 202 per page in the album in accordance to shutterspeed in capturing the images. Then, the main image selecting section212 selects each main image per image capturing condition sorted by theimage sorting section 240. The subsidiary image selecting section 222also selects each subsidiary image per image capturing condition sortedby the image sorting section 240. Thereby, the album generatingapparatus 100 can generate the album having uniformity in each pagebecause it can unify the image capturing condition in each page.

The light source specifying section 256 specifies a type of a main lightsource in capturing the image from the contents of the image of theplurality of images stored in the image storing section 202.Specifically, the light source specifying section 256 specifies the typeof the main light source in capturing the image based on color balancein a range of certain color, i.e., an achromatic color, whose averageluminance in the image is larger than preset luminance. Then, the imagesorting section 240 sorts the plurality of images stored in the imagestoring section 202 per page in the album corresponding to the type ofthe light source specified by the light source specifying section 256.Thereby, the image sorting section 240 can distinguish images capturedunder artificial illumination from images captured under the sun and cansort them to each page. Therefore, the album generating apparatus 100can generate the album having uniformity with respect to a light sourcein each page.

The compositional similarity calculating section 258 calculates a degreeof compositional similarity among the plurality of images stored in theimage storing section 202. Then, the image sorting section 240 sorts theimages calculated to have a high degree of similarity by thecompositional similarity calculating section 258 into the same page inthe album. Then, among the plurality of images sorted by the imagesorting section 240 into the same page, the layout determining section210 lays out the page by using the images calculated to have lowersimilarity by the compositional similarity calculating section 258.Thereby, the album generating apparatus 100 can generate the albumhaving uniformity in terms of composition in each page.

The main subject sorting section 266 sorts the images into each pagebased on a main subject. The composite information storing section 216stores composite information indicating, per page, whether the imagesorted based on the distribution of image capturing times calculated bythe distribution-of-image capturing time calculating section 252 is tobe disposed in the page or the image sorted by the main subject sortingsection 266 based on the main subject is to be disposed in the page.Then, the layout determining section 210 lays out the image sorted bythe image sorting section 240 into the page storing the compositeinformation of the composite information storing section 216 instructingto dispose the image sorted based on the distribution of image capturingtimes into the page and lays out the image sorted by the main subjectsorting section 266 to the page storing the composite information of thecomposite information storing section 216 indicating to dispose theimage sorted by the main subject to the page. Thereby, the albumgenerating apparatus 100 can generate the album occasionally havingpages arranged in a row other than a row arranged in a time-seriesmanner in the album in which pages are arranged in the time-seriesmanner. Thus, the album generating apparatus 100 can generate the albumhaving a story while having an accent.

It is noted that the composite information storing section 216 storesthe composite information, per page, indicating whether the image sortedbased on the distribution of image capturing positions calculated by thedistribution-of-image capturing position calculating section 254 is tobe disposed in the page or the image sorted by the main subject sortingsection 266 based on the main subject is to be disposed in the page.Then, the layout determining section 210 lays out the image sorted bythe image sorting section 240 into the page storing the compositeinformation of the composite information storing section 216 instructingto dispose the image sorted based on the distribution of image capturingpositions to the page and lays out the image sorted by the main subjectsorting section 266 to the page storing the composite information of thecomposite information storing section 216 instructing the image sortedbased on the main subject to the page. Thereby, the album generatingapparatus 100 can generate the album occasionally having pages sorted bythe subject in the album in which the pages are sorted per position forexample. Therefore, the album generating apparatus 100 can generate thealbum having a story while having an accent.

Thus, the album generating apparatus 100 described above can generatethe album having the images adequately sorted and allocated to each pageand having a typical image, selected out of them, as the main image ofeach page. Still more, the album generating apparatus 100 can combinethe main image and the subsidiary image stored in each page in acombination desirable for the appreciator. Accordingly, the user 180 canobtain the desirable album without taking much time.

FIG. 4 is a flowchart showing one exemplary processing flow forgenerating the album. The image sorting section 240 sorts the imagesinto each page in Step S402. The main image selecting section 212selects the main image out of the images sorted in Step S402 into eachpage in Step S404. The subsidiary image selecting section 222 alsoselects the subsidiary image based on the contents of the main image outof the images sorted in Step S402 into each page in Step S406. Then, thelayout determining section 210 lays out the main image selected in Step404 in Step S408. The layout determining section 210 also lays out thesubsidiary image selected in Step S406 in Step S410. It is noted that asfor the procedure for generating the album, the layout determiningsection 210 may determined the number of subsidiary images to be storedbased on the layout of the main image after determining the layout ofthe main image, beside the procedure shown in FIG. 4. Then, the albumgenerating apparatus 100 may select the subsidiary images correspondingto the number of subsidiary images and may lay out the subsidiaryimages.

FIG. 5 is a flowchart showing one exemplary process flow for sorting theimages. FIG. 5 shows the detail of the process in Step S402 for sortingthe images in FIG. 4. The distribution-of-image capturing timecalculating section 252 calculates a distribution of image capturingtimes of the plurality of images in Step S502. Then, thedistribution-of-image capturing position calculating section 254calculates a distribution of image capturing positions of the pluralityof images in Step S504. Then, the importance judging section 260computes deviation of the image capturing position based on thedistribution of image capturing positions calculated in Step S502 inStep S506.

The importance judging section 260 judges whether or not the deviationof the distribution of image capturing positions computed in Step S506is larger than a reference value set in advance in Step S508. When thedeviation of the distribution of image capturing positions is largerthan the reference value set in advance in Step S508, the importancejudging section 260 allocates the images to each page based on thedistribution of image capturing positions in Step S510. When thedeviation of the image capturing position is less than the referencevalue set in advance in Step S508, the importance judging section 260allocates the images to each page based on the distribution of imagecapturing times in Step S512. The images may be allocated to the pageper region where the images were captured or to the page per date whenthe images were captured by the process described above for sorting theimages. Accordingly, the album may be generated in accordance to thecategory desirable for the appreciator.

FIG. 6 shows one exemplary case when the allocation per page by means ofthe distribution of image capturing positions is desirable. FIG. 6 showsa case when an album is to be generated by using images collectivelycaptured in each region of Nagasaki, Hiroshima, Kyoto and Tokyo. In thiscase, while the user captured the images at each visiting place, theuser traveled to the next tourist spot by traveling a distance fullyexceeding a positional range of the image capturing activity in eachregion. In such a case, the importance judging section 260 determinesthat the deviation of the image capturing position is large and judgesthat the image capturing position is important more than the imagecapturing time. Thus, the album generating apparatus 100 can adequatelyallocate the images to the page per region corresponding to the imagecapturing position of the image.

FIG. 7 shows one exemplary case when allocation per page by the imagecapturing time is desirable. FIG. 7 shows a case when the user 180captured images almost in the same region and a period of time duringwhich the images were captured is widely dispersed like a case ofcapturing images for recording growth of a child for example. In thiscase, the importance judging section 260 determines that the deviationof the image capturing position is small and judges that the importanceof the image capturing time is greater than that of the image capturingposition. Thus, the album generating apparatus 100 can sort the imagescaptured to record the child's growth in accordance to the imagecapturing periods and can generate the album in which each page isarranged in order of the image capturing periods. Therefore, the user180 can appreciate the state how the child has been growing every timewhen the user 180 turns the pages and can appreciate the albumpleasantly without feeling any sense of incompatibility. It is notedthat the album generating apparatus 100 may not lay out the imagessorted within the same page based on the distribution of image capturingtimes even in sorting the images based on the distribution of imagecapturing times.

FIG. 8 shows one exemplary method of the importance judging section 260in judging importance of the distribution of image capturing positions.For each of the images stored in the image storing section 202, thedistribution-of-image capturing position calculating section 254calculates a distance between image capturing position where the imagewas captured and image capturing position of an image capturedimmediately before that image as a traveling distance traveled tocapture the images. Then, the distribution-of-image capturing positioncalculating section 254 calculates a distribution of number of imageswith respect to the calculated traveling distance.

Based on the calculated number of images, the importance judging section260 computes a number of images captured at places distant more than thetraveling distance obtained by multiplying a preset factor to a varianceof the traveling distance to compute a ratio of the number of images tothe whole number of images as the deviation of the distribution of imagecapturing positions. Thereby, the importance judging section 260 canjudge that the image capturing position is more important than the imagecapturing time when the user 180 generates an album recording capitalcities of the European countries when the user visited them for example.

FIG. 9 shows one exemplary allocation of images per page based on thedistribution of image capturing times. The image capturing time sortingsection 262 computes a value obtained by dividing a number of capturedimages by a total number of pages of the album as a number of images tobe allocated to one page. For example, the image capturing time sortingsection 262 judges the number of images per page as 4.5 images in caseof FIG. 9. Then, the distribution-of-image capturing time calculatingsection 252 computes intervals of image capturing times of the imagesstored in the image storing section 202 in order of the image capturingtimes.

Then, the image capturing time sorting section 262 sorts the images intoimage groups in order of those having longer intervals between the imagecapturing times, e.g., Δt901, Δt903, Δt902 and so on. For example, theimage group A is separated from the other image groups and is sorted atposition of the image capturing interval Δt901. Then, the imagecapturing time sorting section 262 determines a category in which anumber of images of the sorted image group is almost equal to a numberof images to be allocated to one page and allocates each image group inthe determined category to one and same page. In case of FIG. 9, theimage capturing time sorting section 262 sorts images contained in imagegroups A and B, images contained in an image group C and imagescontained in an image group D, respectively, into one and same page. Itbecomes possible to allocate images captured in nearby time adequatelyto the pages while preventing from being stored in totally separatepages because the image capturing time sorting section 262 sorts theimages into each page based on the distribution of image capturing timesas described above.

FIG. 10 shows one exemplary image allocation per page based on thedistribution of image capturing positions. The image capturing positionsorting section 264 computes intervals of image capturing positions ofthe images stored in the image storing section 202 in order of imagecapturing times. The image capturing time sorting section 262 sorts theimages in order of those having longer intervals between the imagecapturing positions, e.g., D1004, D1002, D1003 and so on in case of FIG.10. For example, the image groups A, B and C are sorted separately fromthe image group D at the position of the image capturing interval D1004.Then, the image capturing time sorting section 262 determines a categoryin which a number of images of the sorted image group is almost equal toa number of images to be allocated to one page and allocates each imagegroup in the determined category to one and same page. In case of FIG.10, the image capturing position sorting section 264 sorts the imagescontained in the image groups A and B, the images contained in the imagegroup C and the images contained in the image groups D, respectively,into one and same page.

FIG. 11 is a flowchart showing one exemplary processing flow forselecting the main image. FIG. 11 shows a detail of the process (S404)for selecting the main image in FIG. 4. The object sampling section 206selects one image out of the images sorted by the image sorting section240 in Step S1102. Then, the object sampling section 206 detects aplurality of objects by sampling contours of a plurality of subjects bymeans of edge sampling or the like out of the selected image and samplesone object among them in Step S1104. The person recognizing section 230then judges whether or not the object sampled in Step S1104 is a personin Step S1106.

When the sampled object is judged to be a person in Step S1106, thenumber-of-object-image calculating section 208 counts up a number ofimages containing the object per person in Step S1108. When it is judgedthat the sampled object is not a person in Step S1106, thenumber-of-object-image calculating section 208 discriminates the sametype of object and counts up a number of images containing the objectper type of the object.

Then, the object sampling section 206 judges whether or not the countingprocess of the number-of-object-image calculating section 208 has beencompleted for all objects contained in one image selected in Step S1102in Step S1112. When it is judged in Step S1112 that the counting processhas not been completed for all of the objects, the process is returnedto Step S1104 to select an other object. When it is judged in Step S1112that the counting process has been completed for all of the objects, itis judged whether or not the process for sampling objects from all ofthe images sorted by the image sorting section 240 has been completed inStep S1114. When it is judged in Step S1114 that the process forsampling from all of the images has not been completed, the process isreturned to Step S1102 to select an other image anew.

When it is judged in Step 1114 that the process for sampling objectsfrom all of the images has been completed, the main image selectingsection 212 discriminates a person or a type of an object counted morethan a reference number of images set in advance among the number ofimages per person or the number of images of objects per type calculatedin Step S1108 or S1110 and selects an image per selected person or animage containing the object contained in the type of the selected objectas a main image in Step S1116. Thus, the album generating apparatus 100can generate the album desirable for the user 180 because itautomatically selects the main image out of the images of the subjectswhich were captured more by the user 180 having concern to that.

FIG. 12 shows one exemplary method for calculating a momentum of anobject. At timing when the user 180 presses a release button, the imagecapturing device 110 captures an image 1222 as well as preliminaryimages 1221, 1223 and so on) at timing before/after that timing. Themomentum calculating section 232 carries out edge sampling or the liketo differential images between the preliminary image 1221 and the image1222 and between the image 1222 and the preliminary image 1223 forexample to sample contours of the object moved while capturing theimages. Then, the momentum calculating section 232 calculates themovement of the object on the image as a momentum of the object. In caseof FIG. 12, motion of both arms of a person, i.e., a subject, remains inthe differential images, so that the momentum calculating section 232discriminates the movement of the both arms. Then, the main imageselecting section 212 selects the image containing the object containingthe subject whose movement is large as a main image. Thereby, the albumgenerating apparatus 100 can select the image containing the subjecthaving a large motion as the main image of the album.

It is noted that the album generating apparatus 100 may calculate themomentum of each sampled object based on relative movement with an otherobject. For example, the momentum calculating section 232 may calculatevariation of relative distance between the both arms of the subject anda mountain on the back in each of the preliminary image 1221 and theimage 1222 as the momentum. Thereby, the album generating apparatus 100can adequately calculate the momentum of the object even when the user180 captured images while panning/tilting the image capturing device110.

It is noted that the image capturing device 110 may record imagecapturing conditions in capturing the images and preliminary images andthe album generating apparatus 100 may calculate the momentum of theobject based on the image capturing conditions. For example, the imagecapturing device 110 records moving speed of focal point of the imagecapturing device 110 in the real space at the timing in capturing theimage and preliminary images. Then, the album generating apparatus 100may calculate the momentum of the object based on the moving speed ofthe focal point in the real space and the movement of the object on theimages judged from the image and preliminary images.

FIG. 13 shows one exemplary method for judging importance of an object.The momentum calculating section 232 calculates a length of a contourline 1302 of the object sampled by the object sampling section 206 andmultiplies a factor proportional to the calculated length with themovement calculated by the momentum calculating section 232. Besidethat, the momentum calculating section 232 may calculate a total lengthof sides of a circumscribing rectangle 1304 circumscribing the objectsampled by the object sampling section 206 as a length of a contour lineof the object. It allows the image containing the subject that occupiesa larger image range to be preferentially selected among movingsubjects, so that the album generating apparatus 100 can generate thealbum desirable for the user 180.

FIG. 14 is a flowchart showing one exemplary processing flow forselecting subsidiary images. FIG. 14 shows a detail of the process forselecting the subsidiary images in Step S406 in FIG. 4. The objectsampling section 206 samples objects in Step S1402. Then, the distancecalculating section 274 calculates a distance D between the objects onthe image or on the real space for each combination of the objectscontained in one image in Step S1404. It is noted that in Step S1404,the distance calculating section 274 may calculate a distance betweenpositions of center of gravity of the sampled objects in the image asthe distance D of the objects in the image.

Then, the number-of-object-image calculating section 208 calculates anumber of images N containing the objects in the same image. Then, theperson recognizing section 230 judges whether or not the objects sampledin Step S1402 are both persons in Step S1408. When it is judged in StepS1408 that the objects are both persons, α1 is set as a factor of personα and when it is judged in Step S1408 that at least one of the objectsis not a person, α2 that is smaller than α1 is set as the factor ofperson α.

Then, the relationship judging section 272 calculates a product of thedistance D calculated in Step S1404, the number of images N calculatedin Step S1406 and the factor of person a set in Step S1410 or S1412 asthe relationship for each of the combination of two objects in StepS1414. The subsidiary image selecting section 222 selects an objectwhose relationship of combination with the object contained in the imageselected as the main image is larger than a reference value ofrelationship set in advance based on the relationship of combination ofthe objects calculated in Step S1414. Then, the subsidiary imageselecting section 222 selects the image containing the selected objectas the subsidiary image in Step S1416. As described above, the albumgenerating apparatus 100 can automatically select the image of a personcaptured together with a person captured in the main image as thesubsidiary image. Accordingly, the user 180 can appreciate the albumwithout feeling a sense of incompatibility.

FIG. 15 is a flowchart showing an other exemplary processing flow forselecting the subsidiary images. FIG. 15 shows an other example of adetail of the process (S406) for selecting the subsidiary images in FIG.4. The person recognizing section 230 judges whether or not a main imageselected by the main image selecting section 212 contains a person inStep S1502. When it is judged in Step S1502 that the main image containsa person, the subsidiary image selecting section 222 selects an imagecontaining a person other than the person in the main image or an imagecontaining no object judged by the person recognizing section 230 as aperson as a subsidiary image in Step S1504.

When it is judged in Step S1502 that the main image contains no person,the subsidiary image selecting section 222 selects an image containing aperson or an image containing a different type of subject as asubsidiary image in Step S1506. Thereby, the subsidiary image selectingsection 222 can prevent the images stored in one and same page frombecoming only images containing the same person or becoming only imagescontaining the same type of subject and can generate the album in whichthe contents of the images stored in each page are well balanced.

FIG. 16 is a flowchart showing a still other exemplary flow forselecting the subsidiary images. The similarity calculating section 242calculates similarity of the main image selected by the main imageselecting section 212 with an other image in Step S1602. Specifically,the similarity calculating section 242 calculates the similarity to besmall for an image having a big time difference from image capturingtime of the main image. Beside that, the similarity calculating section242 may calculate the similarity to be small for an image whose imagecapturing position is largely distant from that of the main image. Stillmore, the similarity calculating section 242 may calculate thesimilarity to be small for an image having different colors from colorscontained in the main image to be small. Further, the similaritycalculating section 242 may calculate the similarity to be small for animage whose focal distance is largely different from that of the mainimage.

Then, the subsidiary image selecting section 222 selects an image whosesimilarity with the main image, calculated in Step S1602, is small as asubsidiary image in Step S1604. Thereby, the subsidiary image selectingsection 222 can select the image having the contents not similar to thatof the main image as the subsidiary image. For example, the albumgenerating apparatus 10 may generate a page in which a landscape imagewhose focal distance is long and a snap shot whose focal distance isshort are combined. Thereby, it becomes possible to prevent the page ofthe album from being composed of only landscape images whose focaldistance is long and from becoming dull.

FIG. 17 is a flowchart showing one exemplary processing flow of thespecific subject information getting section 290. The specific subjectinformation getting section 290 obtains specific subject informationwhich can be used in sorting the images, e.g., in sorting images by thesubject by the main subject sorting section 266, in Step S402 in FIG. 4,in selecting the main image in Step S404 and selecting the subsidiaryimages in Step S406 for example. The specific subject informationgetting section 290 obtains casting information that indicates a role ofa person contained in the image of the album by enabling the user 180 toinput it in Step S1702. For example, the specific subject informationgetting section 290 requests the user 180 to input “Hero” representingthe hero of the album, “Father”, “Mother”, “Brother” and “Sister”representing, respectively, the father, mother, brother and sister ofthe hero, as well as “Friend” representing a friendship with the hero,as casting information.

Then, the specific subject information getting section 290 displays theimages stored in the image storing section 202 to present to the user180 in Step S1704. The specific subject information getting section 290also causes the user 180 to select an image containing a person having arole in the casting information inputted in Step S1702 to obtain theimage in Step S1706. At this time, the specific subject informationgetting section 290 stores a file name of the selected image. Then, thespecific subject information getting section 290 causes the user 180 toinput position of that person in the image selected in Step S1706 inStep S1708. For example, the specific subject information gettingsection 290 may cause the user 180 to input the position of the personby requesting the user 180 to press buttons indicating Left, Center,Right and the like. Then, the specific subject information gettingsection 290 judges whether or not the casting information inputted inStep S1702 is “Hero” in Step S1710. When it is judged in Step S1710 thatthe casting information is “Hero”, the specific subject informationgetting section 290 obtains the name and date of birth of that personfrom the user 180 in Step S1712.

Then, in Step S1714, the specific subject information getting section290 generates specific subject information containing the castinginformation inputted in Step S1702, the file name of the file selectedin Step S1706, the position inputted in Step S1708 and the name and dateof birth obtained in Step S1710. It is noted that when it is judged inStep S1710 that the casting information is not “Hero”, the specificsubject information getting section 290 shifts the process to Step S1714to generate specific subject information containing the castinginformation inputted in Step S1702, the file name of the file selectedin Step S1706 and the position inputted in Step S1708.

Then, the specific subject information getting section 290 asks the user180 whether or not specific subject information of an other person is tobe inputted and judges whether or not casting information of the otherperson is inputted from the input of the user 180 in Step S1716. When itis judged that casting information of the other person is to be inputtedin Step S1716, the specific subject information getting section 290shifts the process to Step S1702 to get the casting information of theother person. When it is judged in Step S1716 that casting informationof the other person is not inputted, the process ends.

As described above, the album generating apparatus 100 obtains the roleof the person in the album by causing the user 180 to specify the personwho is to appear in the album and the role thereof in the album.Therefore, the album generating apparatus 100 can adequately determinethe disposition of the images in the album in accordance to the role ofthe person in the album.

FIG. 18 shows one exemplary contents of the specific subject informationgenerated by the specific subject information getting section 290. Thespecific subject information 1802, 1804 and 1806 are specific subjectinformation specifying, respectively, the hero, a brother of the heroand a friend of the hero in the album. The specific subject information1802, 1804 and 1806 have Name field indicating the name of the person,File field indicating a file name of an image, Position field indicatingposition in the image, Casting field indicating casting information andDate of Birth field indicating date of birth, respectively. Although thespecific subject information of the hero contains individual informationsuch as the name and date of birth, such individual information is notcontained in the specific subject information of the person other thanthe hero. Accordingly, since it is not necessary to input personalinformation of the person other than the hero, there will be almost noproblem even if the data is looked in inputting through a kiosk terminalor even if the data is looked furtively on a communication line ininputting each information contained in the specific subject informationthrough Internet and the like. The specific subject information gettingsection 290 may also generate specific subject information in whichinputted personal information is cryptographed as for the specificsubject information indicating the hero.

FIG. 19 shows exemplary templates stored in the template storing section214. The template 1901 shows one exemplary template of a back and frontcovers of the album and templates 1902 through 1905 show exemplarytemplates of spread pages. The casting information of a person containedin an image to be fitted into each image frame is defined in each imageframe indicated by the templates. For example, it is so defined thatimages containing the person whose Casting field is Hero are fitted intoimage frames 1911 and 1912 indicated in the template 1901. Still more,it is so defined that images containing persons whose Casting field areFather, Mother and Brother, respectively, are fitted into image frames1951 through 1953 in the template 1905.

In selecting the image to be fitted into each image frame, the layoutdetermining section 210 selects specific subject information havingcasting information defined in each image frame. Then, the layoutdetermining section 210 samples an image of person from an image rangeof a predetermined range from the position of Position field in an imagehaving a file name shown in File field contained in the selectedspecific subject information. Then, the layout determining section 210selects an image containing a subject that coincides with the image ofperson with a degree of coincidence more than a predetermined degree ofcoincidence out of the images sorted into that page by the image sortingsection as an image to be fitted into the image frame.

It is noted that ‘relatives’ defined in image frames 1932 through 1935is generic name of casting information indicating family membersincluding Father, Mother, Brother and Sister. Then, the layoutdetermining section 210 fits an image containing a person having eitherone of Father, Mother, Brother and Sister as casting information intothe image frames appointed as ‘relatives’.

Disposition and contents of texts presenting a title of the album andpage titles are defined in the template in advance. For example, as thetext of the album title, <Name>s' Story is defined in the template 1901.Still more, <Name>'s Family and Members are defined respectively to thetemplates 1903 and 1905.

It is noted that the text of the title defined by the template includestext replacement information indicating that at least a part of the textis to be replaced with field data contained in the specific subjectinformation. For example, the text of the title defined in the templates1901 and 1903 include text replacement information <Name> indicatingthat it is to be replaced with Name field. Then, the layout determiningsection 210 generates the text of each title by replacing part of thetext indicated by the text replacement information <Name> by data ofName field specified by the specific subject information whose Castingfield is Hero. Thereby, the album generating apparatus 100 can use thetitle appropriate to the specified hero. As described above, the albumgenerating apparatus 100 can generate the album in which the images aredisposed with the adequate layout corresponding to the subjectinformation specified by the user 180.

FIG. 20 shows exemplary layout results 2002 and 2004 laid out by thelayout determining section 210. The layout determining section 210 fitsthe images into the image frames 1931 through 1945 defined by thetemplate 1903. In the layout result 2002, the layout determining section210 fits an image selected by the main image selecting section 212 as amain image and containing a person whose Casting field is Hero into theimage frame 1931. The layout determining section 210 also fits imagesselected by the subsidiary image selecting section 222 as subsidiaryimages and containing persons whose Casting field is Family into theimage frames 1932 through 1935.

It is noted that the layout determining section 210 may enlarge atrimmed image obtained by trimming the main image to fit it into theimage frame 1931 of the main image so that an area of the hero (personwhose Casting field is Hero) contained in the image frames 1931 through1935 contained in the same page becomes larger than an area of persons(persons whose Casting field is some one other than the hero) other thanthe hero. Still more, the subsidiary image selecting section 222 mayselect subsidiary images to be fitted into the image frames 1932 through1935 so that a number of heroes contained in the image frames 1931through 1935 contained in the same page becomes larger than a number ofpersons other than the hero.

The layout result 2004 is one exemplary layout result when two heroesare specified. At this time, the main image selecting section 212selects an image containing the two persons, i.e., the heroes, as a mainimage. Then, the layout determining section 210 fits the main image intothe image frame 1931. At this time, the main image or the subsidiaryimage may be trimmed so that areas of the plurality of heroes becomealmost equal to fit into the image frames 1931 through 1935. Asdescribed above, the album generating apparatus 100 can dispose theimages with adequate layout corresponding to the role of the subject bythe specific subject information obtained by the specific subjectinformation getting section 290.

FIG. 21 shows one exemplary composite information of images stored inthe composite information storing section 216. The composite informationis defined per page to define which images sorted by which sortingmethod are to be disposed to each page. For example, it is defined tostore images sorted by a main subject to a first spread page (spreadpage No. 1) and a fifth spread page (spread page No. 5) in FIG. 21.Still more, it is defined to store images sorted by time to secondthrough fourth spread pages (spread pages Nos. 2 through 5).

Because image capturing time when an image was captured becomes animportant index in an album recording a child's growth for example, thecomposite information storing section 216 defines composite informationindicating a page sorting method mainly based on time as the compositeinformation. For example, the composite information storing section 216defines composite information indicating so as to dispose the imagessorted by time in the second through fourth spread pages. Then, thelayout determining section 210 allocates the images sorted by the imagecapturing time sorting section 262 based on the distribution of times toeach page in order of average time of the image capturing time of imagessorted into each page. Thereby, the album generating apparatus 100 cangenerate the album having the spread pages in line with the child'sgrowth.

The composite information storing section 216 also defines to disposethe images sorted by the main subject to the first spread page disposedright after the back and front cover pages. In this case, an imagecontaining all persons appearing in the album is disposed in the firstpage of the contents of the album for example. Therefore, theappreciator can understand the whole persons appearing in the album inthe first page of the album for example. Then, because the images sortedby image capturing time for example are disposed in the three spreadpages thereafter, the appreciator can enjoy seeing how the hero hasgrown every time when the appreciator turns the page. Then, imagessorted by the main subject, e.g., a plurality of images containing thewhole family members, are disposed in the fifth spread page for example.Thus the composite information storing section 216 stores the compositeinformation for sporadically inserting the spread pages sorted by themain subject while disposing the images sorted mainly in order of timein the spread pages. Thereby, the album generating apparatus 100 canprevent a monotonous album from being generated and can generate analbum having an accent while having a story.

It is noted that the composite information storing section 216 may storeappropriate composite information per type of an album, beside thegrowth recording album described above, such as a travel recordingalbum, an event recording album and the like. For example, the compositeinformation storing section 216 may store composite information forarranging images in the spread pages mainly based on the distribution ofimage capturing positions. As described above, the composite informationstoring section 216 defines the sorting method adequate to each spreadpage, so that the album generating apparatus 100 can allocate the imagessorted by the adequate sorting method to each spread page in adequateorder.

FIG. 22 shows one exemplary method of the characteristic color samplingsection 226 in sampling a characteristic color. The characteristic colorsampling section 226 calculates each color area, i.e., an area of colorcontained in each of a plurality of color ranges (color ranges A, B, Cand so on) divided into predetermined color ranges in the main imageselected by the main image selecting section 212. It is noted that inthe main image selected by the main image selecting section 212, thecharacteristic color sampling section 226 may calculate a number ofpixels contained in each of the plurality of color ranges as an indexindicating the area.

Then, the characteristic color sampling section 226 calculates an arearatio, i.e., a ratio of the calculated color area to an area of the mainimage, for each of the plurality of color ranges. The characteristiccolor sampling section 226 also stores, in advance, an expected value ofthe area ratio indicating an area ratio expected to appear in an imagefor each of the plurality of color ranges. The expected value of thearea ratio is an expected value per color range contained in an averageimage for example. For example, the characteristic color samplingsection 226 may calculate the area ratio for each of the images storedin the image storing section 202 and to store an average value thereofas the expected value of the area ratio.

Then, the characteristic color sampling section 226 calculates acharacteristic degree that is a ratio of the calculated area ratio tothe expected value of the area ratio per color range to sample a colorrange where the ratio is maximized as a characteristic color in the mainimage. In the example of the figure, the characteristic color samplingsection 226 calculates 1.25, 1.5 and 4 as the characteristic degree ofthe color ranges A, B and C. Thus, the characteristic color samplingsection 226 samples the color range having an area ratio larger than theexpected value as the characteristic color. Accordingly, thecharacteristic color sampling section 226 can adequately sample thecharacteristic color contained in the main image.

It is noted that the template coloring determining section 228 stores aplurality of color palettes showing a combination of colors visuallyharmonized for human. Then, the template coloring determining section228 selects the color palette containing the characteristic colorsampled by the characteristic color sampling section 226 to arrangecolor in the template by using the color contained in the selected colorpalette. For example, the template coloring determining section 228determines colors used for characters of titles contained in thetemplates 1901, 1903 and 1905, background colors of each page and colorsof the image frame defined by the template. Thereby, the albumgenerating apparatus 100 can generate the page well harmonized andadequately colored containing the color close to the characteristiccolor of the main image.

FIG. 23 is a diagram showing one exemplary directory configuration of analbum information file recorded by the album generating apparatus 100 inthe recording medium 160. The album information recording section 270stores an index file 2302, a group information directory 2310 in whichgroup information files 2311 through 2319 describing group informationand an image directory 2320 right under a route directory 2300. Thealbum information recording section 270 also stores group imagedirectories 2330, 2340, 2350 and so on storing the images sorted by theimage sorting section 240 per page right under the image directory 2320.The album information recording section 270 also stores groupinformation files 2332, 2342 and 2352 right under the group imagedirectories 2330, 2340 and 2350.

The album information recording section 270 stores images 2334 a through2334 z sorted respectively by the image capturing time sorting section262 and the group index file 2332 as well as images 2344 a through 2344z and the group image information file 2342 under the group imagedirectories 2330 and 2340. Then, the album information recording section270 stores a group image information file 2352 and images sorted by themain subject sorting section 266 right under the group image directory2350. Here, the album information recording section 270 stores not theimage file it self but link information indicating a link to imagesstored in the other group image directory right under the group imagedirectory 2350. This link information may be described in the groupimage information file 2352 or may be recorded by a symbolic link andthe like to the images stored in the other group image directory.Therefore, the image itself will not be recorded while overlapping fromeach other when the images are sorted by different sorting methods.Accordingly, the album generating apparatus 100 can store the imagesefficiently in the recording medium 160.

It is noted that the group image information files 2332, 2342 and 2352describe layout information by which each of the images stored in thedirectory of the same level is outputted. For example, the group imageinformation files 2332, 2342 and 2352 describe printing layoutinformation such as position and size of images printed in a spread pageof the album as well as color and the like of the template to beprinted. The group image information files 2332, 2342 and 2352 alsodescribe display layout information such as position, size order ofimages displayed in a same display area in an electronic album as wellas color and the like of the template.

The group information files 2311, 2312 and 2313 describe informationspecifying the group image information files 2332, 2342 and 2352, e.g.,file names of the group image information files 2332, 2342 and 2352. Theindex file 2302 describes information specifying the group informationfiles 2311 through 2319, e.g., file names of the group information files2311 through 2319. The index file 2302 also defines order for outputtingthe images managed by the group image information files 2332, 2342 and2352 corresponding to the group information files 2311, 2312 and 2313.For example, the index file 2302 describes a page ID for identifying aspread page in which the images managed by the group image informationfiles 2332, 2342 and 2352 are printed in the album. Still more, theindex file 2302 describes a sequence ID for identifying order fordisplaying images managed by the group image information files 2332,2342 and 2352 corresponding to the group information files 2311, 2312and 2313 in the album.

Accordingly, the album generating apparatus 100 specifies the file namein the group information files 2311 through 2319 by reading the indexfile 2302 described in the recording medium 160. Then, the albumgenerating apparatus 100 specifies the file name of the group imageinformation files 2332, 2342, 2352 and soon by reading the groupinformation files 2311 through 2319. Further, the album generatingapparatus 100 can output the images as the album by reading the groupimage information files 2332, 2342 and 2352.

It is noted that the index file 2302, group information files 2311through 2319 and the group image information files 2332, 2342 and 2352may be described in XML format. For example, the index file 2302describes the file names representing the group information files 2311through 2319 as tag data for the tag name for identifying each group. Asdescribed above, the album generating apparatus 100 can determine adisplaying mode of the grouped images by reading the index recorded inthe route directory 2300. Accordingly, the album generating apparatus100 can record the images and the displaying mode of the images in thealbum in the recording medium 160 with the flexible directoryconfiguration.

FIG. 24 is a block diagram showing one exemplary block configuration ofthe album generating apparatus 100 according to an other embodiment. Thealbum generating apparatus 100 of the present embodiment provides analbum generating apparatus for generating an album having an adequateand uniform layout. An environment for using the album generatingapparatus 100 of the present embodiment is the same with the environmentfor using the album generating apparatus 100 explained in connectionwith FIG. 1, so that its explanation will be omitted here. Specifically,the album generating apparatus 100 provides the user 180 with an albumhaving a sense of uniformity by disposing a posterization image to eachpage of the album or by disposing images of related scenes to each page.

The album generating apparatus 100 has an image storing section 2402, analbum generating section 2408, a template storing section 2414, acolor-to-be-used determining section 2426, an image-to-be-used-in-albumselecting section 2420, an posterization image selecting section 2450, aposterization image generating section 2452, a selection conditionstoring section 2454, a characteristic value storing section 2456 and aspecific subject information getting section 2490. Theimage-to-be-used-in-album selecting section 2420 has a main imageselecting section 2412, a subsidiary image selecting section 2422, acompositional similarity calculating section 2430, a color similaritycalculating section 2432, an image capturing environment similaritycalculating section 2434, a time similarity calculating section 2436, animage capturing place specifying section 2438 and a relationship judgingsection 2472. The specific subject information getting section 2490 hasa main person inputting section 2492 and a main person specifyingsection 2494.

The image storing section 2402 stores a plurality of images. Thespecific subject information getting section 2490 obtains an image of amain person in the album. The main person inputting section 2492requests one to input an image containing the main person in the albumand position of the main person in the image out of the plurality ofimages stored in the image storing section 2402. The main personspecifying section 2494 specifies the image of the main person in thealbum out of the plurality of images stored in the image storing section2402. Specifically, the main person specifying section 2494 specifiesthe image of the main person from the image containing the main personand the position thereof inputted through the main person inputtingsection 2492. It is noted that the specific operation and function ofthe main person inputting section 2492 and the main person specifyingsection 2494 are almost same with those of the main person inputtingsection 292 and the main person specifying section 294, so that theirexplanation will be omitted here.

The main person image sampling section 2496 samples a plurality ofimages of the main person, i.e., subject images of the main person, outof the subject images contained in the plurality of images stored in theimage storing section 2402 by comparing the image of the main personspecified by the main person specifying section 2494 with the subjectimages contained in the plurality of images stored in the image storingsection 2402. Then, the posterization image selecting section 2450selects an image suitable for a posterization process out of theplurality of images of the main person sampled by the main person imagesampling section 2496. The posterization image generating section 2452applies the posterization process to the image selected by theposterization image selecting section 2450 to generate a posterizationimage to be used in the album. Then, the album generating section 2408generates the album containing the image of the main person by using theposterization image generated by the posterization image generatingsection 2452.

The selection condition storing section 2454 stores selecting conditionsfor selecting the image suitable for posterization. Specifically, theselection condition storing section 2454 stores the selecting conditionspresenting an extent of a distribution of luminance suitable for theposterization image. Then, the posterization image selecting section2450 selects a posterization image that agrees with the selectingcondition stored in the selection condition storing section 2454.Specifically, the posterization image selecting section 2450 selects animage having an extent of distribution of luminance that conforms to theextent of the distribution of luminance stored in the selectioncondition storing section 2454.

The posterization gradation determining section 2440 determinesgradation of the posterization process. It is noted that the selectioncondition storing section 2454 stores the selecting conditions pergradation of posterization. Then, the posterization image selectingsection 2450 selects a posterization image that conforms to theselecting condition stored in the selection condition storing section2454 by correlating with the gradation of the posterization imagedetermined by the posterization gradation determining section 2440. Theposterization image generating section 2452 generates the posterizationimage to be used in the album by applying the posterization with thegradation determined by the posterization gradation determining section2440 to the image selected by the posterization image selecting section2450.

The color-to-be-used determining section 2426 determines each charactercolor to be used for characters defined in the template of each page ofthe album. Then, the posterization image generating section 2452generates a posterization image of the character color determined by thecolor-to-be-used determining section 2426 by applying the posterizationprocess to the image selected by the posterization image selectingsection 2450. It is noted that when there is a more number of colorsused for the characters determined by the color-to-be-used determiningsection 2426, the posterization gradation determining section 2440 maydetermine the gradation of the posterization process to be large. Thealbum generating section 2408 generates an album containing a page laidout so that the posterization image generated by the posterization imagegenerating section 2452 adjoins to the characters for which thecharacter color determined by the color-to-be-used determining section2426 in each page. Thereby, the posterization image having a sense ofuniformity with the title characters of each page may be generated.

It is noted that the image-to-be-used-in-album selecting section 2420may select the plurality of images to be used for the album out of theplurality of images stored in the image storing section 2402. Then, themain person image sampling section 2496 samples the plurality of imagesof the main person, i.e., the subject images of the main person, out ofthe subject images contained in the plurality of images selected by theimage-to-be-used-in-album selecting section 2420. The posterizationimage selecting section 2450 selects the image suitable forposterization out of the plurality of images of the main person sampledby the main person image sampling section 2496.

The characteristic value storing section 2456 stores a characteristicvalue of the posterization image used in the album generated by thealbum generating section 2408 and the posterization image generatingsection 2452 generates a posterization image having a characteristicvalue different from any one of characteristic values of theposterization images stored in the characteristic value storing section2456 when the album generating section 2408 generates a new album. Then,the album generating section 2408 generates the new album by using theposterization images generated by the posterization image generatingsection 2452.

It is noted that the album generating section 2408 disposes oneposterization image generated by the posterization image generatingsection 2452 to a plurality of pages of the album. Specifically, thealbum generating section 2408 disposes one posterization image generatedby the posterization image generating section 2452 to the front and backcover pages of the album as well as to predetermined position of thepages composing the album. Therefore, the posterization image used ineach album becomes unique per album. Therefore, since the albums may beidentified by the posterization image, the posterization image may beused for schedule control in dealing with re-ordering of the albums andin the production of the albums.

The main image selecting section 2412 selects the main image that is tbe the primary image in the album out of the plurality of images storedin the image storing section 2402. The relationship judging section 2472judges the relationship between the main image selected by the mainimage selecting section 2412 and the plurality of images stored in theimage storing section 2402 based on the contents of the plurality ofimages stored in the image storing section 2402. Then, the subsidiaryimage selecting section 2422 selects the image judged by therelationship judging section 2472 that it is highly related with themain image selected by the main image selecting section 2412 as asubsidiary image that is to be subsidiary to the main image selected bythe main image selecting section 2412 in the album out of the pluralityof images stored in the image storing section 2402. Then, the layoutdetermining section 2410 lays out the main image selected by the mainimage selecting section 2412 and the subsidiary image selected by thesubsidiary image selecting section 2422 in the same page in the albumbased on the contents of the main image selected by the main imageselecting section 2412 and the subsidiary image selected by thesubsidiary image selecting section 2422.

The template storing section 2414 stores a judging condition of highlyjudging the relationship of the main image with an image whose imagecapturing time is closer to image capturing time of the main image.Then, the relationship judging section 2472 highly judges therelationship of the main image with the image whose image capturing timeis closer to that of the main image.

The template storing section 2414 stores a judging condition of highlyjudging the relationship of the main image with an image whosecomposition is more similar to the main image. The compositionalsimilarity calculating section 2430 calculates a degree of similarity ofcompositions of the images based on composition of subject contained inthe plurality of images stored in the image storing section 2402. Then,the relationship judging section 2472 may highly judge the relationshipof the main image with the image whose composition is calculated to behighly similar to that of the main image by the compositional similaritycalculating section 2430.

The template storing section 2414 also stores a judging condition ofhighly judging the relationship of the main image with an imagecontaining a color that is more similar to that of the main image. Thecolor similarity calculating section 2432 judges the degree ofsimilarity of the colors of the images based on a distribution of colorscontained in the plurality of images stored in the image storing section2402. Then, the relationship judging section 2472 highly judges therelationship of the main image with the image whose color was calculatedto be more similar to that of the main image.

The template storing section 2414 stores a judging condition of highlyjudging the relationship of the main image with an image whose imagecapturing environment is similar to that of the main image. The imagecapturing environment similarity calculating section 2434 calculates thesimilarity of image capturing environments in which the plurality ofimages stored in the image storing section 2402 were captured. Then, therelationship judging section 2472 highly judges the relationship of themain image with an image whose image capturing environment was judged bythe image capturing environment similarity calculating section 2434 tobe highly similar to that of the main image.

The template storing section 2414 stores a judging condition of highlyjudging the relationship of the main image with an image whose imagecapturing place is similar to that of the main image. The imagecapturing place specifying section 2438 specifies whether the placewhere the plurality of images stored in the image storing section 2402was captured was indoor or outdoor based on the contents of the imagesstored in the image storing section 2402. Then, the relationship judgingsection 2472 highly calculates the relationship of the main image withan image whose image capturing place was specified to be same with thatof the main image by the image capturing place specifying section 2438more than the relationship of the main image with an image whose imagecapturing place was specified to be different from that of the mainimage by the image capturing place specifying section 2438.

The template storing section 2414 stores a judging condition of highlyjudging the relationship of the main image with an image whose imagecapturing time is close to that of the main image. The image capturingtime here is a concept representing timing when the image was captured,such as a season, a month and time in a day. The time similaritycalculating section 2436 judges closeness of the image capturing timesof the plurality of images stored in the image storing section 2402.Then, the relationship judging section 2472 highly judges therelationship of the main image with the image whose image capturing timewas calculated by the time similarity calculating section 2436 to becloser to that of the main image.

Thus, the template storing section 2414 stores the judging conditions ofjudging the degree of relationship between the main image and thesubsidiary image. Then, based on the judging conditions stored in thetemplate storing section 2414, the relationship judging section 2472judges the degree of relationship between the main image and theplurality of images stored in the image storing section 2402. Thus, thealbum generating apparatus 100 can generate the album having a sense ofuniformity throughout the album because it disposes the sameposterization image in each page of the album. Still more, because thealbum generating apparatus 100 uses the image of the hero afterundergoing through the posterization process, it can generate the uniquealbum in accordance to intention of the user 180. Still more, becausethe album generating apparatus 100 selects the subsidiary image relatedwith the main image, the appreciator may feel a sense of uniformity interms of contents in each page.

FIG. 25 is a drawing showing one exemplary posterization image. Theposterization image generating section 2452 generates a posterizationimage 2514 by applying the posterization process to an image 2512 of ahero contained in an image 2502. The posterization image may be an imagein which a pixel value of the image is limited to a discrete value andit may be an image in which a regional image of the hero is binalized orthree-valued. It is noted that the posterization image may not belimited to the image in which the regional image of the hero isbinalized or three-valued but may be an image to which the posterizationprocess is applied to a degree recognizable to be different from theimage.

FIG. 26 shows one exemplary disposition of the posterization imagedefined by the template storing section 2414. The template storingsection 2414 describes that the posterization image is disposed so as toadjoin with the page title for impressing the album title of the frontand back covers composing the album and the theme of the page composingthe album to the appreciator of the album as posterization imagedisposition information. For example, posterization image dispositioninformation 2610 of a cover page 2600 of the album defines so as to layout a posterization image having an almost same height with that of thealbum title at the position adjoining to the album title at the upperpart of the cover face overlapped with the front cover. Similarly tothat, posterization image disposition information 2611 of the template2601 of the back and front covers defines so as to lay out aposterization image at almost same height with the album title so as toadjoin to the album title at the upper part of a face that becomes afront cover of the album. Still more, posterization image dispositioninformation 2612 and 2613 of templates 2602 and 2603 of first and secondpages composing the contents of the album define so as to lay out aposterization image at almost same height with the height of the albumtitle at the position adjoining to page titles (Casting, Be ambitious!)at the upper right part of the page region of a spread page.

Then, the album generating section 2408 disposes the posterization imageat position and size corresponding to the position of the posterizationimage disposition information 2610 through 2613 defined by the templatestoring section 2414. It is noted that the posterization imagedisposition information 2610 through 2613 may define so as to disposethe posterization image having a same width with a height of text suchas the adjoining album title and page title. The posterization imagedisposition information 2610 through 2613 also define so as to use thesame color with the color used for the text such as each titledetermined by the color-to-be-used determining section 2426. Theposterization image disposition information 2610 through 2613 may alsodefine so as to use a posterization image having gradation correspondingto the gradation of text such as each title determined by thecolor-to-be-used determining section 2426. Still more, the posterizationimage disposition information 2610 through 2613 may define so as to usea posterization image having gradation corresponding to a number ofcolors of the text such as each title determined by the color-to-be-useddetermining section 2426. The posterization image generating section2452 may generate the posterization image having the color and gradationdefined by the posterization image disposition information.

As described above, since the album generating apparatus 100 disposesthe same posterization image adjoining to each title of the album, itcan provide the user 180 with the album having a sense of uniformity asa whole. It is noted that desirably, the image of the hero generatingthe posterization image is selected out of the images used in the album.The appreciator can pleasantly appreciate the album generated by thealbum generating apparatus 100 by selecting it out of the images used inthe album.

FIG. 27 is a graph showing one exemplary distribution of luminance ofimages. The posterization image selecting section 2450 calculates aluminous distribution of the image of the main person sampled by themain person image sampling section 2496. Luminous distributions 2701 and2702 are exemplary luminous distributions calculated by theposterization image selecting section 2450 from different images of themain person sampled by the main person image sampling section 2496.

Then, the posterization image selecting section 2450 calculates adivergent index B, that is a rate of a number of pixels having aluminous value less than a reference luminance a set in advance to awhole number of pixels and a divergent index B_(β) that is a rate of anumber of pixels having a luminous value less than a reference luminanceβ set in advance to a whole number of pixels. Then, the posterizationimage selecting section 2450 selects an image whose divergent indicesB_(α) and B_(β) are larger than the reference value as an image forgenerating a posterization image. It is noted that the selectioncondition storing section 2454 may store the reference luminance α and βas well as minimum values B_(αmin) and B_(βmin) of the divergent indicesB_(α) and B_(β) corresponding to the reference luminance α and β. Then,the posterization image selecting section 2450 calculates the divergentindices B_(α) and B_(β) based on the reference luminance α and β storedin the selection condition storing section 2454 and the image of themain person selected by the main person image sampling section 2496. Theposterization image selecting section 2450 then selects an image whosecalculated divergent indices B_(α) and B_(β) is larger than thereference value of the divergent indices B_(α) and B_(β) stored in theselection condition storing section 2454 as an image suitable forgenerating a posterization image. It is noted that the posterizationimage selecting section 2450 may preferentially select an image whosedivergent indices B_(α) and B_(β) are largest among the images of themain person selected by the main person image sampling section 2496 andwhose divergent indices B_(α) and B_(β) are larger than the referencevalues B_(αmin) and B_(βmin) set in advance as an image suitable for theposterization image.

As described above, the posterization image selecting section 2450 canselect the image of the main person whose contrast is clear as an imageto which the posterization process is applied. For example, theposterization image selecting section 2450 judges that an image of theluminous distribution 2702 is suitable for the posterization image morethan an image of the luminous distribution 2701 in the case of thefigure. Accordingly, it is possible to prevent a most area of theposterization image generated by the posterization process from beingwhited out or from being blacked out in advance. It is noted that besidethe combinations of reference luminance α and β and the divergentindices B_(αmin) and B_(βmin), the selection condition storing section2454 may store a condition of selecting an image containing a face of aperson as an image suitable for a posterization image more than an imagecontaining no face of a person. At this time, the selection conditionstoring section 2454 may store a condition of selecting an imagecontaining a face of a person seen more from the front side as an imagesuitable as a posterization image. Thereby, the posterization imagegenerating section 2452 can generate a posterization image copyingfeatures of a hero such that the appreciator can recognize the hero byconnecting with the posterization image.

It is noted that the selection condition storing section 2454 may storereference luminance and divergent index corresponding to gradation inwhich the posterization image generating section 2452 carries out theposterization process. Although the selection condition storing section2454 stores the condition that the binary posterization process is basedon a number of pixels contained in the two regions of minimum luminanceto reference luminance α and reference luminance β to maximum luminancein the case described above, it stores a condition that the three-valuedposterization process is based on a number of pixels contained in threeregions of minimum luminance to reference luminance α, referenceluminance γ to reference luminance η and reference luminance β tomaximum luminance for example. Thereby, the selection condition storingsection 2454 can provide the posterization image selecting section 2450with the adequate selecting condition corresponding to variousgradations such as three-valued process.

FIG. 28 is a table showing exemplary characteristic values of theposterization image stored in the characteristic value storing section2456. The characteristic value storing section 2456 stores patterninformation and color information of posterization images by correlatingwith posterization image IDs for identifying a posterization imagegenerated by the posterization image generating section 2452. Forexample, when the posterization process is binary, the characteristicvalue storing section 2456 stores position of a pixel whose luminance ismaximum luminance, e.g., 1, or minimum luminance, e.g., 0, as patterninformation. Beside that, the characteristic value storing section 2456may store information of the position of the pixel having the maximum orminimum luminance coded corresponding to the position of the pixel aspattern information. The characteristic value storing section 2456 alsostores values of color information, e.g., each component of R, G and B,in the pixel whose luminance is maximum luminance for example.

Thus, the characteristic value storing section 2456 stores thecharacteristic values of the posterization image already generated bythe posterization image generating section 2452. Then, when theposterization image generating section 2452 generates a posterizationimage anew, it generates the posterization image having differentcharacteristic values from the characteristic values stored in thecharacteristic value storing section 2456. Therefore, the albumgenerating apparatus 100 can generate the new posterization image havingthe different characteristic values from the posterization image alreadygenerated and use in an album generated anew. Accordingly, the albumgenerating apparatus 100 can provide the user 180 with own posterizationimage. Still more, the user 180 can use the posterization image inrenewing an order. For example, a photo shop can readily specify analbum whose order is to be renewed when the user 180 brings the album tothe photo shop or transmits an image of the posterization image to thephoto shop by a portable phone or the like. The posterization image maybe also used for production control as an identification mark of eachalbum by printing the posterization image in the cover, in the backcover and front cover pages and in the page of the contents of the albumas explained in connection with FIG. 26. It is noted that theposterization image generating section 2452 may generate theposterization image as two-dimensional code such as QR code.

FIG. 29 shows one exemplary judging condition stored in the templatestoring section 2414. A template 2900 of one spread page contains pagetitle information 2911 defining position, size and contents of the pagetitle, posterization image disposition information 2912 where aposterization image is disposed, an image frame 2901 defining so that amain image is fitted into it and image frames 2902 through 2906 definingso that subsidiary images are fitted into them. In this case, the layoutdetermining section 2410 determines to fit the main image selected bythe main image selecting section 2412 to the image frame 2901.

It is also defined so as to fit an image captured at the same imagecapturing place with that of the image fitted into the image frame 2901for the main image into the image frame 2902 based on the judgingcondition of the image capturing place. It is also defined so as to fitimages captured at almost same image capturing time with the imagefitted into the image frame 2901 for the main image into the imageframes 2903, 2904, 2905 and 2906 defined by the template based on thejudging condition of the image capturing time. Thus, the templatestoring section 2414 stores the judging conditions for judging therelationship of the image to be fitted into the image frame of thesubsidiary image with the main image.

Therefore, the subsidiary image selecting section 2422 can select asubsidiary image having a relationship that conforms to the conditionfor judging the relationship with the main image defined in thetemplate. For example, images of scenes temporarily continuous with thetime when the main image was captured may be disposed in the imageframes 2903 through 2906 and an image of a scene positionally andtemporarily continuous with the main image may be disposed in the imageframe 2902. Therefore, the album generating apparatus 100 can preventimages having totally no relationship with the main image from beingdisposed. It is noted that the template storing section 2414 stores thepage title 2911 conforming to the relational condition. As describedabove, the album generating apparatus 100 can dispose the imagescontaining scenes related to each other in the same spread page.Therefore, the album generating apparatus 100 enables the images toconform to an image disposing image.

FIG. 30 is a block diagram showing one exemplary hardware configurationof a computer 1500 of the album generating apparatus 100. The computer1500 of the present embodiment has a CPU peripheral section having a CPU1505, a RAM 1520, a graphic controller 1575 and a display device 1580mutually connected by a host controller 1582, an input/output sectionhaving a communication interface 1530, a hard disk drive 1540 and aCD-ROM drive 1560 connected with the host controller 1582 via aninput/output controller 1584 and a legacy input/output section having aROM 1510, a flexible disk drive 1550 and an input/output chip 1570connected with the input/output controller 1584.

The host controller 1582 connects the RAM 1520, the CPU 1505 thataccesses the RAM 1520 at high transfer rate and the graphic controller1575. The CPU 1505 operates based on programs stored in the ROM 1510 andthe RAM 1520 to control the respective sections. The graphic controller1575 obtains image data generated by the CPU 1505 and others on a framebuffer provided within the RAM 1520 to display on the display device1580. Instead of that, the graphic controller 1575 may contain the framebuffer for storing the image data generated by the CPU 1505 and others.

The input/output controller 1584 connects the host controller 1582 withthe communication interface 1530, which is a relatively fastinput/output device, the hard disk drive 1540 and the CD-ROM drive 1560.The hard disk drive 1540 stores programs and data used by the CPU 1505within the computer 1500. The communication interface 1530 communicateswith the album generating apparatus 100 via network to provide programsor data to the album generating apparatus 100. The CD-ROM drive 1560reads the program or data out of the CD-ROM 1595 and provides it to thehard disk drive 1540 and to the communication interface 1530 via the RAM1520.

The input/output controller 1584 is connected with the relatively slowinput/output devices of the ROM 1510, the flexible disk drive 1550 andthe input/output chip 1570. The ROM 1510 stores a boot program executedby the computer 1500 in starting the system and programs and the likedependent on the hardware of the computer 1500. The flexible disk drive1550 reads a program or data out of a flexible disk 1590 and provides itto the hard disk drive 1540 and to the communication interface 1530 viathe RAM 1520. The input/output chip 1570 connects the flexible diskdrive 1550 and the various input/output devices via parallel ports,serial ports, keyboard ports, mouse ports and the like.

The program provided to the communication interface 1530 via the RAM1520 is stored in the flexible disk 1590, the CD-ROM 1595 or therecording medium such as an IC card to be provided to the user. Theprogram is read out of the recording medium to be provided to thecommunication interface 1530 via the RAM 1520 and to be transmitted tothe album generating apparatus 100 via the network. The programtransmitted to the album generating apparatus 100 is installed in andexecuted by the album generating apparatus 100.

The program installed to and executed by the album generating apparatus100 operates the album generating apparatus 100 so as to function as theimage storing section 202, the object sampling section 206, thenumber-of-object-image calculating section 208, the layout determiningsection 210, the main image selecting section 212, the subsidiary imageselecting section 222, the trimming section 224, the person recognizingsection 230, the momentum calculating section 232, the area-to-contourratio calculating section 234, the image sorting section 240, thesimilarity calculating section 242, the distribution-of-image capturingtime calculating section 252, the distribution-of-image capturingposition calculating section 254, the importance judging section 260,the relationship judging section 272, the distance calculating section274, the specific subject information getting section 290, theimage-of-main person selecting section 296, the characteristic colorsampling section 226, the template coloring determining section 228, thetemplate storing section 214, the composite information storing section216, the light source specifying section 256, the compositionalsimilarity calculating section 258 and the album information recordingsection 270 explained in connection with FIGS. 1 through 23. The programinstalled to and executed by the album generating apparatus 100 operatesthe image sorting section 240 so as to function as the image capturingtime sorting section 262 and the image capturing position sortingsection 264 and operates the layout determining section 210 so as tofunction as the disposition determining section 282 and the sizedetermining section 284. Still more, the program installed to andexecuted by the album generating apparatus 100 causes the specificsubject information getting section 290 to function as the main personinputting section 292 and the main person specifying section 294. Theprogram installed to and executed by the album generating apparatus 100explained in connection with FIGS. 22 through 29 operates the albumgenerating apparatus 100 so as to function as the image storing section2402, the album generating section 2408, the template storing section2414, the color-to-be-used determining section 2426, theimage-to-be-used-in-album selecting section 2420, the posterizationimage selecting section 2450, the posterization image generating section2452, the selection condition storing section 2454, the characteristicvalue storing section 2456 and the specific subject information gettingsection 2490 explained in connection with FIGS. 22 through 29. Stillmore, the program operates the image-to-be-used-in-album selectingsection 2420 so as to function as the main image selecting section 2412,the subsidiary image selecting section 2422, the compositionalsimilarity calculating section 2430, the color similarity calculatingsection 2432, the image capturing environment similarity calculatingsection 2434, the time similarity calculating section 2436, the imagecapturing place specifying section 2438 and the relationship judgingsection 2472 explained in connection with FIGS. 22 through 29 and causesthe specific subject information getting section 2490 to function as themain person inputting section 2492 and the main person specifyingsection 2494 explained in connection with FIGS. 22 through 29.

The program described above may be stored in an external storage medium.As the storage medium, an optical recording medium such as DVD and PD, amagneto-optic recording medium such as MD, a tape medium and asemiconductor memory such as IC card may be used beside the flexibledisk 1590 and the CD-ROM 1595. Still more, a storage device such as ahard disk and RAM provided in a server system connected with a dedicatedcommunication network and Internet may be used as the recording mediumand the program may be provided to the communication line 1500 via thenetwork.

Although the invention has been described by way of the exemplaryembodiments, it should be understood that those skilled in the art mightmake many changes and substitutions without departing from the spiritand scope of the invention.

It is obvious from the definition of the appended claims that theembodiments with such modifications also belong to the scope of theinvention.

1. An album generating apparatus, comprising: an image storing sectionfor storing a plurality of images; a main image selecting sectionconnected to the image storing section for selecting a main image thatis to be a primary image in an album out of the plurality of imagesstored in said image storing section; a subsidiary image selectingsection connected to the image storing section for selecting an imagerelated with the main image out of the plurality of images stored insaid image storing section as a subsidiary image that is to besubsidiary to the main image in the album selected by said main imageselecting section; and a layout determining section connected to themain image selecting section and the subsidiary image selecting sectionfor laying out the main image selected by said main image selectingsection and the subsidiary image selected by said subsidiary imageselecting section in the album based on contents of the main imageselected by said main image selecting section and the subsidiary imageselected by said subsidiary image selecting section.
 2. The albumgenerating apparatus as set forth in claim 1, further comprising: anobject sampling section for sampling a plurality of objects contained ineach of the plurality of images stored in said image storing section;and a relationship judging section for judging that a relationship levelof a first object with a second object is high when a number of imagescontaining said first and second objects sampled by said object samplingsection is larger than a reference number set in advance in theplurality of objects stored in said image storing section, wherein thesubsidiary image selecting section selects the image containing saidsecond object that has been judged by said relationship judging sectionto be highly related with said first object contained in said main imageselected by said main image selecting section as a subsidiary image tobe subsidiary to the main image in the album selected by said main imageselecting section.
 3. The album generating apparatus as set forth inclaim 2, further comprising: a distance calculating section forcalculating distance between said first and second objects in the imagecontaining said first and second objects; wherein said relationshipjudging section judges that the relationship level of said first objectwith said second object is high when the distance calculated by saiddistance calculating section is short.
 4. The album generating apparatusas set forth in claim 2, further comprising: a distance calculatingsection for calculating distance between said first and second objectsin actual space; wherein said relationship judging section judges thatthe relationship level of said first object with said second object ishigh when the distance calculated by said distance calculating sectionis short.
 5. The album generating apparatus as set forth in claim 2,further comprising: a person recognizing section for recognizing whetheror not an object sampled by said object sampling section is a person;wherein said relationship judging section judges a relationship level ofobjects recognized as persons by said person recognizing section to behigher than a relationship of objects recognized as not persons by saidperson recognizing section.
 6. An album generating method, comprising:an image storing step of storing a plurality of images; a main imageselecting step of selecting a main image that is to be a primary objectin an album out of the plurality of images stored in said image storingstep; a subsidiary image selecting step of selecting an image relatedwith the main image out of the plurality of images stored in said imagestoring step as a subsidiary image that is to be subsidiary to the mainimage in the album selected in said main image selecting step; and alayout determining step of laying out the main image selected in saidmain image selecting step and the subsidiary image selected in saidsubsidiary image selecting step in the album based on contents of themain image selected in said main image selecting step and the subsidiaryimage selected in said subsidiary image selecting step.
 7. The albumgenerating method as set forth in claim 6, further comprising: an objectsampling step of sampling a plurality of objects contained in each ofthe plurality of images stored in said image storing step; and arelationship judging step of judging that a relationship level of afirst object with a second object is high when a number of imagescontaining said first and second objects sampled in said object samplingstep is larger than a reference number set in advance in the pluralityof objects stored in said image storing step, wherein the subsidiaryimage selecting step selects the image containing said second objectthat has been judged by said relationship judging section to be highlyrelated with said first object contained in said main image selected insaid main image selecting step as a subsidiary image to be subsidiary tothe main image in the album selected in said main image selecting step.8. A computer readable medium storing thereon a program for an albumgenerating apparatus for generating an album, operating said albumgenerating apparatus so as to function as: an image storing section forstoring a plurality of images; a main image selecting section forselecting a main image that is to be a primary object in the album outof the plurality of images stored in said image storing section; asubsidiary image selecting section for selecting an image related withthe main image in the album selected by said main image selectingsection; and a layout determining section for laying out the main imageselected by said main image selecting section and the subsidiary imageselected by said subsidiary image selecting section in the album basedon contents of the main image selected by said main image selectingsection and the subsidiary image selected by said subsidiary imageselecting section.
 9. The computer readable medium as set forth in claim8, the program operating said album generating apparatus so as tofurther function as: an object sampling section for sampling a pluralityof objects contained in each of the plurality of images stored in saidimage storing section; and a relationship judging section for judgingthat relationship level of a first object with a second object is highwhen a number of images containing said first and second objects sampledby said object sampling section is larger than a reference number set inadvance in the plurality of objects stored in said image storingsection, wherein the subsidiary image selecting section selects theimage containing said second object that has been judged by saidrelationship judging section to be highly related with said first objectcontained in said main image selected by said main image selectingsection as a subsidiary image to be subsidiary to the main image in thealbum selected by said main image selecting section.
 10. The albumgenerating apparatus as set forth in claim 1, further comprising: arelationship judging section for judging a relationship of the mainimage selected by said main image selecting section with the pluralityof images stored in said image storing section based on contents of theplurality of images stored in said image storing section, wherein thesubsidiary image is judged by said relationship judging section to behighly related with the main image selected by said main image selectingsection, and the main image selected by said main image selectingsection and the subsidiary image selected by said subsidiary imageselecting section are laid out in one and same page in the album basedon the contents of the main image selected by said main image selectingsection and the subsidiary image selected by said subsidiary imageselecting section.
 11. The album generating apparatus as set forth inclaim 10, further comprising: a template storing section for storing ajudging condition for judging the degree of relationship between themain image and the subsidiary image; wherein said relationship judgingsection judges the degree of relationship between the main image and theplurality of images stored in said image storing section based on thejudging conditions stored in said template storing section.
 12. Thealbum generating apparatus as set forth in claim 11, wherein saidtemplate storing section stores a judging condition of highly judgingthe relationship of the main image with an image whose image capturingtime is closer to that of the main image; and said relationship judgingsection highly judges the relationship of the image whose imagecapturing time is closer to that of the main image.
 13. The albumgenerating apparatus as set forth in claim 11, further comprising: acompositional similarity calculating section for calculating the degreeof similarity of composition of the images based on composition ofsubjects contained in the plurality of images stored in said imagestoring section; wherein said template storing section stores a judgingcondition of highly judging the relationship of the main image with theimage whose composition is similar to that of the main image; and saidrelationship judging section highly judges the relationship of the mainimage with the image whose composition is calculated to be highlysimilar to that of the main image by said compositional similaritycalculating section.
 14. The album generating apparatus as set forth inclaim 11, further comprising: a color similarity calculating section forjudging the degree of similarity of colors of the plurality of imagesstored in said image storing section based on distributions of colorscontained therein; wherein said template storing section stores ajudging condition of highly judging the relationship of the main imagewith an image containing color therein is similar to that of the mainimage; and said relationship judging section highly judges therelationship of the main image with the image whose color is calculatedto be highly similar to that of the main image by said color similaritycalculating section.
 15. The album generating apparatus as set forth inclaim 11, further comprising: a similarity-of image capturingenvironment calculating section for calculating similarity of imagecapturing environments in which the plurality of images stored in saidimage storing section were captured; wherein said template storingsection stores a judging condition of highly judging the relationship ofthe main image with the image whose image capturing environment issimilar to that of the main image; and said relationship judging sectionhighly judges the relationship of the main image with the image whoseimage capturing environment is calculated to be highly similar to thatof the main image by said similarity-of image capturing environmentcalculating section.
 16. The album generating apparatus as set forth inclaim 11, further comprising: an image capturing place specifyingsection for specifying whether the places where the plurality of imagesstored in said image storing section were captured are indoor or outdoorbased on contents of the plurality of images stored in said imagestoring section; wherein said template storing section stores a judgingcondition of highly judging the relationship of the main image with animage whose image capturing place is similar to that of the main image;and said relationship judging section highly calculates the relationshipof the image, with the main image, whose image capturing place isspecified to be the same with that of the main image by said imagecapturing place specifying section more than the relationship of animage, with the main image, whose image capturing place is specified tobe different from that of the main image by said image capturing placespecifying section.
 17. The album generating apparatus as set forth inclaim 11, further comprising: a time similarity calculating section forjudging closeness of image capturing times of the plurality of imagesstored in said image storing section; said template storing sectionstores a judging condition of highly judging the relationship of themain image with the image whose image capturing time is close to that ofthe main image; and said relationship judging section highly judges therelationship of the main image with the image whose image capturing timeis calculated to be closer to that of the main image by said timesimilarity calculating section.
 18. The album generating method as setforth in claim 6, further comprising: a relationship judging step ofjudging a relationship of the main image selected in said main imageselecting step with the plurality of images stored in said image storingstep based on contents of the plurality of images stored in said imagestoring step, wherein the subsidiary image is judged by saidrelationship judging step to be highly related with the main imageselected in said main image selecting step, and the main image selectedin said main image selecting step and the subsidiary image selected insaid subsidiary image selecting step are laid out in one and same pagein the album based on the contents of the main image selected in saidmain image selecting step and the subsidiary image selected in saidsubsidiary image selecting step.
 19. The computer readable medium as setforth in claim 8, the program operating said album generating apparatusso as to further function as: a relationship judging section for judginga relationship of the main image selected by said main image selectingsection with the plurality of images stored in said image storingsection based on contents of the plurality of images stored in saidimage storing section, wherein the subsidiary image is judged by saidrelationship judging section to be highly related with the main imageselected by said main image selecting section, and the main imageselected by said main image selecting section and the subsidiary imageselected by said subsidiary image selecting section are laid out in oneand same page in the album based on the contents of the main imageselected by said main image selecting section and the subsidiary imageselected by said subsidiary image selecting section.